Misusing the transformations can have a detrimental impact on significance testing, confidence intervals, reported standard errors of correlations, and meta-analysis.

The tutorial is available at

http://www.minerazzi.com/tutorials/nonadditivity-correlations-part-2.pdf

Filed under: Correlation Coefficients, Data Mining, IR Tutorials, Mathematics, meta-analysis, self-weighting, Statistics and Mathematics Tagged: Correlation Coefficients, Data Mining, Mathematics, Pearson Correlation, Spearman Correlation, statistics, tutorials ]]>

http://www.minerazzi.com/tools/regression-correlation/calculator.php

The tool does simple linear regression and correlation analyses, computing Spearman and Pearson correlation coefficients and other relevant statistics.

The companion default example was intentionally selected to illustrate that for rank data free from ties, Spearman and Pearson correlation coefficients are the same thing.

Filed under: calculators, Correlation Coefficients, Data Mining, Mathematics, Software, Statistics and Mathematics Tagged: Pearson Correlation, Regression, Simple Linear Regression, Spearman Correlation, statistics ]]>

http://www.minerazzi.com/causality/

Find resources or build curated collections about causality, causation, and determinism.

Filed under: Causality & Determinism, Curated Collections, Data Mining, IR Tools, Mathematics, miner, minerazzi, Statistics and Mathematics Tagged: Causality, Causation, Determinism, Mathematics, miner, minerazzi, statistics ]]>

The article is available at

http://minerazzi.com/tutorials/nonadditivity-correlations-part-1.pdf

07-04-2017 update: In page 1 the line for the *Beta _{1 }*should read “is the slope of a simple linear regression model”. My fault. Fixed today along with few other nuances.

Enjoy it!

Filed under: Correlation Coefficients, Data Mining, IR Tutorials, Marketing Research, Mathematics, Statistics and Mathematics Tagged: Correlation Coefficients, Data Mining, Mathematics, statistics, tutorials ]]>

http://www.minerazzi.com/tools/t-p-effect-size/estimator.php.

This tool estimates t-scores from p-values and vice versa for a given number of degrees of freedom υ. Just enter a (t,υ) or (p,υ) pair and this tool will solve for the missing term.

The tool also estimates the statistics that one would obtain if the computed estimates correspond to a set of paired variables (x, y), or to effect sizes from any two samples of same sizes (n1 and n2); i.e. samples with same degrees of freedom (υ1 and υ2).

The tool’s page lists some interesting exercises and good references for effect size conversions and meta-analysis in general. Enjoy it!

Filed under: calculators, Data Mining, Mathematics, meta-analysis, Statistics and Mathematics Tagged: effect sizes, meta-analysis, statistical analysis, statistics, Student's t test ]]>

http://www.minerazzi.com/tools/student-t-table/generator.php

This tool allows you to easily generate a customized table of Student’s t-values.

That is done by iteratively calling (i.e., looping) the very same algorithm that we use for our t-values Calculator.

This tool comes handy when you don’t have statistical *t*-tables around or are working with *p* and *t* values, or degrees of freedom, not available from such tables. Avoid pausing a problem for annoying linear interpolation workarounds!

Filed under: calculators, Data Mining, Mathematics, Software, Statistics and Mathematics Tagged: Mathematics, statistics, Student's t test ]]>

http://www.minerazzi.com/meta-analysis/

Find resources relevant to meta-analysis, effect sizes, and power analysis with this Minerazzi miner.

Build curated collections on these subjects.

The miner includes Bing and Google RSS News channels powered by our SPP tool.

Filed under: Data Mining, Human-Computer Interaction, IR Tools, Mathematics, meta-analysis, News, Physiology, Queries, Scripts, Software, Statistics and Mathematics Tagged: effect sizes, meta-analysis, minerazzi, miners, power analysis ]]>

http://www.minerazzi.com/tools/t-values/calculator.php

The tool works by computing, through numerical approximation methods, the Inverse Cumulative Distribution Function or ICDF (also known as the Quantile Function, QF) of the Student’s t Distribution.

This new tool complements our p-values Calculator, described in

https://irthoughts.wordpress.com/2017/06/12/p-values-calculator-for-students-t-hypothesis-testing/

and available at

http://www.minerazzi.com/tools/p-values/calculator.php

The t-values calculator comes handy when you don’t have around statistical t-tables or want to t-testing at confidence levels and degrees of freedom not listed in such tables. In fact, with a small change to the script, we can build another tool for the generation of t-tables at any confidence level and degrees of freedom as specified by a user.

Both tools will eventually be used to provide a third tool: The t-p Transformation tool that, as the name suggests, transforms CDF (t-to-p) into ICDF (p-to-t) results and vice versa.

Filed under: Algorithms, Data Mining, Mathematics, Software, Statistics and Mathematics Tagged: Mathematics, statistical analysis, statistics, Student's t test, t-values ]]>

http://www.minerazzi.com/tools/p-values/calculator.php

Submitting a Student’s t value and degrees of freedom returns a p-value. This is a great tool for Student’s t hypothesis testing.

The tool works by numerically approximating the CDF (Cumulative Distribution Function). This is the integral of the PDF (Probability Distribution Function) of the Student’s t Distribution. The theory behind these calculations along with valuable references are given in the tool’s page.

PS.

The reverse process, computing t from a p-value is possible by inverting the CDF to compute the Quantile Function (QF), also known as the inverse CDF. Our (soon to be released) t-p Transformations tool computes both the CDF (*t*-to-*p*) and QF (*p*-to-*t*).

Filed under: Algorithms, calculators, Data Mining, Software, Statistics and Mathematics Tagged: hypothesis testing, Mathematics, statistical analysis, statistics, Student's t test, tools ]]>

http://www.minerazzi.com/webspam

http://www.minerazzi.com/zika

http://www.minerazzi.com/usc

http://www.minerazzi.com/seominer

http://www.minerazzi.com/religions

http://www.minerazzi.com/shopper

http://www.minerazzi.com/rblogs

http://www.minerazzi.com/quantum

Filed under: Data Mining, Human-Computer Interaction, Programming, Quantum Computing, Spam Tagged: data miners, minerazzi, miners, Search ]]>

http://aidblab.cse.iitm.ac.in/cs625/

Happy to see that the following readings from the course are based on two articles we published long time ago at Mi Islita.com, now a search engine site about Puerto Rico resources.

Reading #10

Vector Space Models

http://aidblab.cse.iitm.ac.in/cs625/10.VectorSpace-model.pdf

Reading #11

SVD-LSI

http://aidblab.cse.iitm.ac.in/cs625/11.SVD-LSI.pdf

Filed under: Algorithms, Data Mining, ir, IR Tutorials, Latent Semantic Indexing, Mathematics, Software, SVD Tagged: Algorithms, Data Mining, lsi, Mathematics, Singular Value Decomposition, SVD, Vector Space Models ]]>

http://nullege.com/codes/show/src@g@e@gensim-HEAD@gensim@test@test_miislita.py

`This module replicates the miislita vector spaces from`

`"A Linear Algebra Approach to the Vector Space Model -- A Fast Track Tutorial"`

`by Dr. E. Garcia...`

` `

Great and positive accomplishment!

That tutorial is no longer at miislita.com, but was long ago moved to minerazzi.com. Find it here:

http://www.minerazzi.com/tutorials/term-vector-linear-algebra.pdf

For other resources do a search for python in our IR miner at

For inquiries about that implementation, contact its author.

For other inquiries, applications, suggestions, drop me a line.

PS. Please note that Nullege.com itself is a search engine for finding python code. Here is a good example: http://nullege.com/codes/search/wx.calendar.CalendarCtrl

Filed under: Algorithms, Crawlers, Data Mining, inverted index, ir, IR Tools, Open Source Projects, Programming, Queries, Search Engines Architecture Course, Software, SVD, Vector Space Models Tagged: Algorithms, Data Mining, Mathematics, open source, Python, R, Search, tutorials ]]>

The Public Records Database miner (PUBDBS) at

http://www.minerazzi.com/pubdbs/

has been updated and reindexed.

Search by states, gov agencies, or keywords.

Use it to build your own curated collection of public databases.

Find news relevant to public databases.

Filed under: Data Mining, Homeland Security, information retrieval, People Searches, Public Databases, Public Records Tagged: Government Databases, Most Wanted, People Searches, Public Databases, Public Records ]]>

http://www.minerazzi.com/tools/acid-base-systems/calculator.php

The tool replaces a previous one (removed from the web) where only simple acid-base solutions were considered.

The current one is not limited to the analysis of one solution or to acid-base titrations. Actually the tool also supports pH calculations of systems not undergoing titrations.

This is a great tool for real world applications like pH calculations from natural acid-base systems where a sample might consist of multiple acids mixed with several bases and/or their salts.

Examples of such systems are bio-samples (blood, saliva…) as well as geo-samples (sea and river water) and urban samples (sewage, wastewater,..).

This new tool is based on a general equation for acid-base systems that is solved using a residual analysis algorithm, avoiding the need for using inversion techniques, successive approximations, Newton-Raphson method, or the evaluation of derivatives or high-degree polynomials.

Filed under: Algorithms, calculators, chemical mining, chemistry, Data Mining, Mathematics, Software Tagged: chemical mining, chemistry, chemistry mining, chemistry tools ]]>

has been updated and reindexed. Additional records will be added soon.

The miner also features news results from Bing, Google, and other sites.

Use it to build your own curated collection of open software or to access latest news on open projects from around the world.

Filed under: Data Mining, IR Tools, Open Source Projects, Programming, Software, Technology Inventions Tagged: minerazzi, miners, open projects, open software, open source ]]>

http://www.minerazzi.com/tools/activity-coefficients/calculator.php

If you are into chemical data mining, this post might interest you.

This new Minerazzi tool easily solves activity coefficient models for a missing term.

Thus the tool allows you to cross-map activity coefficients to ionic strengths and vice versa using the following models:

Debye-Hückel Limiting Law

Debye-Hückel Extended Limiting Law

Güntelberg

Davis

Truesdell-Jones

The tool , written in PHP, was inspired in the WATEQF program originally written in FORTRAN (https://www.nrc.gov/docs/ML0331/ML033170395.pdf).

Recent versions of that program are available for download from the U.S. Geological Survey site at https://wwwbrr.cr.usgs.gov/projects/GWC_chemtherm/software.htm

Chemistry researchers, teachers and students can benefit from this tool, which is part of an ongoing effort of making available online scientific research tools across different disciplines.

04-05-2017 Update: Additional content and literature references added.

Filed under: Algorithms, calculators, chemical mining, chemistry, Data Mining, Programming, Software Tagged: activity coefficient models, activity coefficients, chemical mining, chemistry, chemistry calculators, chemistry tools, computational chemists, electrochemistry ]]>

http://www.minerazzi.com/cannabis

Find business, research, technology, legal resources, and more relevant to medical cannabis.

Talking about the subject, according to a CNBC news at

http://www.cnbc.com/2017/03/16/oxford-university-to-launch-medical-cannabis-research-programme.html

and quote:

“Oxford University will be at the forefront of a multimillion-pound research program, which hopes to help develop new therapies for acute and chronic conditions by examining the effects of medical cannabis. ”

“The oldest university in England will be teaming up with private equity company, Kingsley Capital Partners who will provide up to £10 million ($12.36 million) in initial investment, which will be funded through its new biopharmaceutical firm Oxford Cannabinoid Technologies (OCT). ”

“The partnership is expected to bring together Kingsley’s financial backing and Oxford’s clinical expertise and research in immunology, neuroscience and cancer, to find ways of developing new treatments for those suffering with pain, cancer and inflammatory disease. ”

“Scientists at the university expect to do this by studying the cellular, molecular, and systems mechanisms of cannabinoids. ”

Filed under: Data Mining, Medical Cannabis, News Tagged: marihuana medicinal, Medical Cannabis, medical marijuana, minerazzi, miners ]]>

Body Mass Index (BMI)

http://www.minerazzi.com/tools/body-mass-index/calculator.php

Corpulence Index (CI)

http://www.minerazzi.com/tools/corpulence-index/calculator.php

Membrane Potentials

http://www.minerazzi.com/tools/membrane-potentials/calculator.php

Cell Electrode Potentials

http://www.minerazzi.com/tools/cell-electrode-potentials/calculator.php

Standard Electrode Potentials

http://www.minerazzi.com/tools/standard-electrode-potentials/calculator.php

Filed under: calculators, chemistry, Data Mining, Physiology Tagged: chemistry, chemistry tools, Physiology, physiology tools, tools ]]>

we mentioned a design pattern for easily cross-mapping all experimental variables from a given model. By applying the pattern to the Nernst Equation, we were able to develop the Standard Electrode Potentials Calculator which cross-maps all 4 experimental terms from said equation. This tool is available at

http://www.minerazzi.com/tools/standard-electrode-potentials/calculator.php

In this post we just want to mention that by applying the same design pattern to the Goldman-Hodgkin-Katz (GHK) Equation we were able to develop the Membrane Potentials Calculator, which cross-maps 12 experimental terms. This new tool is available now at

http://www.minerazzi.com/tools/membrane-potentials/calculator.php

Examples on how to do the mapping of variables are given in the Suggested Exercises section of the tool.

Overall, we are confident that the patterns used in the design of these tools can be applied, across disciplines, to mathematical models consisting of more experimental variables.

Filed under: Algorithms, calculators, chemical mining, chemistry, Data Mining, IR Tools, Mathematics, Programming, Software Tagged: chemistry, cross-mapping, Data Mining, Mathematics, variables ]]>

Information Retrieval, http://www.minerazzi.com/irc

Data Structures & Algorithms, http://www.minerazzi.com/dsac

Both include RSS news channels to Bing, Google, MIT, and Arxiv so users can easily find news relevant to these collections.

Filed under: Algorithms, Curated Collections, Data Mining, Data Structures, information retrieval, ir, IR Tools Tagged: Algorithms, Data Mining, Data Structures, information retrieval, ir, minerazzi, miners ]]>

The tool is available at

http://www.minerazzi.com/tools/standard-electrode-potentials/calculator.php

The tool easily solves the Standard Electrode Potential equation for a missing term. That is, given all but one of the experimental terms, the tool solves for the missing one.

There is no doubt that cross-mapping variables from a single tool set up is more engaging than single-mapping them: A great instructive tool for students and teachers alike.

Enjoy it.

Filed under: calculators, chemical mining, chemistry, Data Conversion, Data Mining Tagged: chemistry, chemistry tools, chemistry tutorials, electrochemistry ]]>

http://www.minerazzi.com/tools/reagent-solutions/maker.php

The tool is not really relevant to information retrieval, but the algorithm that runs behind it describes a useful pattern applicable to many fields and disciplines, including IR.

Anyway, the tool is aimed at solving the following problem.

An undiluted reagent solution is typically prepared by dissolving an amount of solid reagent in a solvent, usually water. Additional solvent is then added to bring the solution up to a specific volume.

With the reagent formula weight the solution concentration can be expressed in moles per liter, after proper unit conversions. A set of undiluted solutions at specific concentration intervals can be prepared in this way, but the calculations can be tedious. This tool was developed to simplify all these calculations.

The tool computes the amount of solid reagent needed to prepare n number of undiluted solutions at specific concentration intervals, ∆C, and up to a maximum concentration level, C, and volume, V, where C > ∆C > 0.

Notice that the set of solutions to be prepared, n, is not known in advance.

Filed under: Algorithms, calculators, chemical mining, chemistry, Data Mining, Scripts, Software Tagged: chemistry, chemistry calculators, chemistry tools ]]>

The Self-Weighting Model (SWM) consists in computing local and global weights from the constituent independent variables of a function and, from these, weighted averages for said function. See

http://www.tandfonline.com/doi/abs/10.1080/03610926.2011.654037

**Model Advantages**

SWM makes possible

1. within-set and between-set comparisons

2. calculation of weighted averages from non-additive quantities

3. acceptance or rejection of candidate weighted averages

4. identification of cases where meta-analysis models and traditional transformations fail

The above is possible by considering variability information (i.e., fluctuations) present in the constituent independent variables of a function. If this information is not available, the model suggests the harmonic mean, a statistic that frequently arises in Science and Engineering, as the candidate weighted average.

**Practical Applications**

With SWM, weighted averages can be easily computed from non-additive quantities like

1. correlation coefficients

2. coefficients of variations

Other applications for SWM are possible.

**Tutorials**

To learn more about SWM you may want to read the following tutorials:

http://www.minerazzi.com/tutorials/self-weighting-model-tutorial-part-1.pdf

http://www.minerazzi.com/tutorials/self-weighting-model-tutorial-part-2.pdf

Filed under: Algorithms, Correlation Coefficients, Data Mining, Mathematics, self-weighting, Statistics and Mathematics Tagged: Correlation Coefficients, Mathematics, meta-analysis, self-weighting, statistics, SWM ]]>

http://www.minerazzi.com/tools/domain/extractor.php

The tool extracts domains and subdomains from up to 10,000 URLs at once. Larger sets are resized to conform to this limit. This is done to avoid browser crashes.

From the input set, the Domain Extractor returns a set consisting of domains and subdomains. The results are deduplicated and sorted in alphabetical order

The tool comes handy when one wants to extract chunks consisting of 10,000 domains from databases or other sources.

It can be conveniently used in combinations of other of our tools, like

The FQU Bot

http://www.minerazzi.com/tools/fqu/fqu.php

and

MUST

http://www.minerazzi.com/tools/must/must.php

Simple, light, but a powerful toy/tool: The Domain Extractor can be used as part of a crawling strategy: Once domains and subdomains are extracted, the chunks of URLs can be sent to a queue for crawlers to revisit them.

Another application consists in querying a search engine, extract URLs from its results page and then process them through the tool.

There might be other applications, but the above can give you an idea of how handy the tool can be.

Filed under: Algorithms, Crawlers, Data Conversion, Data Mining, IR Tools, Programming, Scripts, Software Tagged: Algorithms, Domain Tools, domains extractor, extraction tools, text mining, tools ]]>

Another step closer to MR is provided below:

http://journals.plos.org/plosbiology/article?id=10.1371/journal.pbio.1002593

The abstract states and quote

“Despite partial success, communication has remained impossible for persons suffering from complete motor paralysis but intact cognitive and emotional processing, a state called complete locked-in state (CLIS). Based on a motor learning theoretical context and on the failure of neuroelectric brain–computer interface (BCI) communication attempts in CLIS, we here report BCI communication using functional near-infrared spectroscopy (fNIRS) and an implicit attentional processing procedure. Four patients suffering from advanced amyotrophic lateral sclerosis (ALS)—two of them in permanent CLIS and two entering the CLIS without reliable means of communication—learned to answer personal questions with known answers and open questions all requiring a “yes” or “no” thought using frontocentral oxygenation changes measured with fNIRS. Three patients completed more than 46 sessions spread over several weeks, and one patient (patient W) completed 20 sessions. Online fNIRS classification of personal questions with known answers and open questions using linear support vector machine (SVM) resulted in an above-chance-level correct response rate over 70%. Electroencephalographic oscillations and electrooculographic signals did not exceed the chance-level threshold for correct communication despite occasional differences between the physiological signals representing a “yes” or “no” response. However, electroencephalogram (EEG) changes in the theta-frequency band correlated with inferior communication performance, probably because of decreased vigilance and attention. If replicated with ALS patients in CLIS, these positive results could indicate the first step towards abolition of complete locked-in states, at least for ALS.”

End of the quote.

This article is also quoted at the Wyss Center site:

A Spanish news derivative version can be read from

The Wyss Center also has a neurotech challenge:

http://www.wysscenter.ch/en/neurotech-challenge/

so MR going mainstream is a matter of time.

As mentioned many times before, Mind Retrieval is a promisory information retrieval paradigm. See our previous posts on this topic below:

https://irthoughts.wordpress.com/2016/08/05/artificial-neurons-ibm-and-mind-retrieval/

https://irthoughts.wordpress.com/2016/04/28/a-step-closer-to-mind-retrieval/

https://irthoughts.wordpress.com/2015/06/17/say-hello-to-mind-retrieval/

https://ithinksearch.wordpress.com/2010/08/10/ideas-entrevista-a-edel-garcia/

Filed under: Data Mining, Human-Computer Interaction, IR Tools, Mind Retrieval, News Tagged: Algorithms, Data Mining, Human-Computer Interaction, Mind Retrieval, search engines ]]>

http://www.minerazzi.com/chemsubstances/

Find elements, compounds, ions, and alloys by names, symbols, formulas, & uses. Find information about elements of the Periodic Table. Includes NIST data.

We want to take this opportunity to mention of the following updates:

Hundreds of new records pointing to NIST chemical databases have been added to the Chemical Databases miner available at

http://www.minerazzi.com/chemdbs

Hundreds of new records pointing to Spanish exams have been added to the Chemistry Exams miner available at

http://www.minerazzi.com/chemexams

However, many. (though not all) of these spanish records are pdf files from Spain with no index keywords, so to find them just query the name of the country or a given region. I know, this is a bit odd. You may also try searching for something like química, exámen and so forth with(out) punctuation.

In addition, more records have been added to the following chemistry miners:

http://www.minerazzi.com/chem

http://www.minerazzi.com/chemsoft

http://www.minerazzi.com/chemjs

As Chemistry is a broad topic, expect additional updates to the above or new miners. Who knows? We may eventually need to place them all in a separate host.

Have a great chemistry day!

Filed under: chemical mining, Curated Collections, Data Mining Tagged: chemical databases, chemistry, Chemistry Exams, chemistry journals, chemistry miner, chemistry tools, chemistry tutorials ]]>

http://www.minerazzi.com/chemexams

Find chemistry exams, practice tests, assessment studies, & more.

Great for chemistry teachers and their students.

Use it to find practice tests with(out) answers, standardized tests, and course-specific exams, or build chemistry exams curated collections. Whether you are taking analytical, organic, physical, or general chemistry, this miner is for you.

Litmus test yourself!

Filed under: chemical mining, chemistry, Curated Collections, Data Mining Tagged: chemistry, Chemistry Exams, Chemistry Exams Miner, chemistry miner, chemistry tools ]]>

http://www.minerazzi.com/patents

We also added rss news channels (Google, Bing, and European Patent).

One of the most interesting rss news gathered through the Bing channel is shown below:

Amazon patents highway network to stop self-driving cars crashing

*Wed, 18 Jan 2017 07:09:00 GMT*

Amazon has been awarded a patent for a technology that would enable driverless cars and trucks to navigate reversible lanes.

Patenting a highway network for autonomous vehicles: They are way ahead of business competitors.

Filed under: Data Mining, Patents, Software Tagged: minerazzi, miners, Patents, rss news ]]>

Use it to find resources relevant to 4D Printing. As with all of our miners, you may use it to recrawl individual search results and build your own curated collections on this fascinating new disruptive technology.

Search Examples: Do a search with the miner for the following keywords: [ polymers ], [ printing materials ], and so forth.

**Why a miner on 4D Printing?**

Good question. Here is why: If you think that 3D printing is hot, think again because 4D Printing is here to rock the World!

4D Printing is right now revolutionizing many industries, fields, and disciplines: manufacturing, construction, medical research, biopharma, architecture, design, the arts, engineering, and computing, to mention a few.

**What is 4D Printing?**

4D Printing is one form of Programmable Matter (PM) technology. According to Wikipedia, the “programmable matter” term was coined in 1991 (1).

Skylar Tibbits, through his 2010 MS thesis “Logic Matter: digital logic as heuristics for physical self-guided assembly” (2), established the basis for a new technology: 4D Printing. Tibbits is a young Research Scientist, at MIT Department of Architecture and Director of the Self-Assembly Lab (3) at MIT. He is also credited of inventing the term “4D Printing”. I’m right now reading his fascinating thesis in pdf format. Simply awesome.

A light definition of 4D printing: Objects 3D-printed that evolve in time under environmental stimuli. In other words, 4D Printing is 3D printing with Time as the additional dimension.

So 4D-printed objects can evolve in time and change its molecular structures or mechanical, electrical, optical, or magnetic properties. All the changes are done without human intervention, but with environmental stimuli. Imagine building things that build themselves in the presence of a solvent (e.g, water), light, temperature changes, etc.

I guess you got the idea as to why, as this novel technology is impacting so many industries and sciences, MIT’s Self-Assembly Lab has the following partners and sponsors, among others:

http://www.bmw.com/

http://www.pg.com/

http://www.google.com/

http://www.converse.com/

http://www.suncorp.com/au

http://www.kinginnovation.com/

http://www.steelcase.com/

http://www.carbitex.com/

http://www.autodesk.com/

http://www.airbus.com/

http://www.seedmediagroup.com/

and many more.

**What is Self-Assembly?**

Tibbits and co-workers define this process as follows (3):

“Self-Assembly is a process by which disordered parts build an ordered structure through local interaction. We have demonstrated that this phenomenon is scale-independent and can be utilized for self-constructing and manufacturing systems at nearly every scale. We have also identified the key ingredients for self-assembly as a simple set of responsive building blocks, energy and interactions that can be designed within nearly every material and machining process available. Self-assembly promises to enable breakthroughs across every applications of biology, material science, software, robotics, manufacturing, transportation, infrastructure, construction, the arts, and even space exploration. The Self-Assembly Lab is working with academic, commercial, nonprofit, and government partners, collaborators, and sponsors to make our self-assembling future a reality. ”

**Programmable Matter Defined**

As stated by Campbell, Tibbits, and Garrett in the May, 2014 report of the Atlantic Council of the United States (4, 5):

“Programmable matter (PM) is the science, engineering, and design of physical matter that has the ability to change form and/or function (shape, density, moduli, conductivity, color, etc.) in an intentional, programmable fashion. PM may come in at least two forms: *(1)* objects made of pre-connected elements that are 4D printed or otherwise assembled as one complete structure for self-transformation, and* (2)* unconnected voxels that can come together or break apart autonomously to form larger programmable structures. PM encompasses, yet goes beyond, a range of technological capabilities–including 3D printing, micro-robotics, smart materials, nanotechnology, and micro-electromechanical systems (MEMS), to name a few. ”

**Useful Applications for 4D Printing**

There are many. As noted by Achuth Rao (6):

“4D printing allows the manufacture of objects that transform with time, essentially allowing objects to be “programmed” to behave in certain manner. The prospects of such programmable materials are numerous:”

“On space missions astronauts can take 4D printed objects with them which can transform into desired objects/structures despite the harshness of space.”

“For plumbing & sewage structures, 4D printed pipes will have the ability to expand or contract depending on the amount of water passing through them.”

“In medicine, 4D printing could reduce surgical procedures by enabling doctors to inject self-transforming materials into the body.”

**From Pixels to Voxels**

A voxel is a volumetric pixel that defines the fundamental unit of digital space and programmable matter. These can be both digital (computational representation in 3D models) and physical (consisting of raw materials like integrated circuits, biomaterials, micro-robotics, titanium, nanomaterials, etc).

**From Programming the Matter to Hacking the Matter: New Security Challenges**

4D Printing opens the door to a new science: Programming the Matter. It also opens the door to new challenges: Hacking the Matter. That is to say that if matter can be programmed to change in time, it could also be hacked.

Hacking programmable matter presents new challenges to governments and the society in general. Imagine someone trying to hack a material that evolves in time so it becomes a different shape or adopts different characteristics in time and space. Imagine that once hacked the new shape becomes something else that could disrupt communications, services, or cause harm to humans. Three example suffices:

(a) hacking a programmable PM drug dispenser inside a human body so it dispenses more than the expected or morphs into a weapon.

(b) changing parts of a morphable transportation vehicle (car, train, airplane).

(c) modifying morphable components of clothing, engineered parts, etc.

PM technologies in the hands of terrorists or hostile governments is indeed a very bad thing as can be concluded from the Atlantic Council report (4, 5). In the right hands, as mentioned before, PM technologies and 4D Printing in particular, is set to rock the World!

PM + AI

Imagine Programmable Matter, by means of 4D Printing or other PM technologies, with an Artificial Intelligence component that under the morphological changes in time is also a self-learner, capable of making decisions…

To learn more about this incredible technology and research field, visit the Self-Assembly Lab site (3).

References

1. https://en.wikipedia.org/wiki/Programmable_matter

2. http://dspace.mit.edu/handle/1721.1/64566

3. http://www.selfassemblylab.net/

Filed under: 4D Printing, Algorithms, Curated Collections, Data Mining, Dynamics, Machine Learning, Mathematics, News, Programming, Software, Technology Inventions Tagged: 4D Printing, minerazzi, miners, Programmable Matter, Self-Assembly ]]>

http://www.minerazzi.com/tools/must/must.php

The tool now automatically detects bogus http status code responses. These types of response codes are frequently designed, though not always, to game crawlers and automated header request tools; i.e., to believe that a resource is not accessible.

For instance, test the following with our tool and as given:

arxiv.org

ask.com

webcrawler.com

metacrawler.com

etc…

Have fun!

Filed under: Algorithms, Crawlers, Data Mining, IR Tools, Programming, Software Tagged: Algorithms, link checker, minerazzi tools, redirection checker, tools ]]>

Tired of misleading claims? Even in Science you can find some of these puppies. Here are three of them:

**1. Handling Evidence in History: the Case of Einstein’s Wife.**

Historical claims in science need to be inspected carefully, just as when testing hypotheses. This is a fascinating article by Alberto A. Martínez on claims regarding Einstein’s wife. Many still claim that she co-authored his famous papers. Read it at

**2. Madelung’s Rule and the Sloppy Version of the Aufbau Principle.**

Misconceptions about the Madelung Rule can be traced back to the years after World War Two (WW2). Before WW2, Madelung and others discussed orbital occupations, not orbital energies. After WW2, his rule was somehow absorbed by chemistry textbook authors, suggesting then various graphic mnemonics. The chemists then, incorrectly, interpreted the ‘Madelung graphic’ as representing the orbital energy order in cations of a given element. As a result of this, entire generations of chemistry teachers and their students have been misled. Read more at

http://www.minerazzi.com/tutorials/rydberg-rule-mnemonic.pdf

**3. The Most Influential Paper that Gerard Salton Never Wrote.**

This one is a great example of sloppy peer reviewers and editors alike. Read it at

Filed under: Data Mining, News, Quack Science Tagged: chemistry, Misleading Claims, news, Physics, Quack Science ]]>

To view it you need a browser that supports modern CSS rules.

CSS Animation Credits: http://codepen.io/heyvian/pen/oihDk

This animation consists of minor tweaks we added to their original work.

Please give proper credits if you want to reuse or modify it.

The classic splash interface is still accessible through the new one.

Have some LIGO

Filed under: Data Mining, LIGO, Mathematics Tagged: astronomy, Data Mining, LIGO, ligo miner, Mathematics, minerazzi ]]>

http://www.minerazzi.com/quantum

Find resources relevant to quantum computing, searches, retrieval, and information assurance.

Access from introductory to advanced research papers and how-to articles. This 2017, move beyond classic IR and computing stuff and forward to new research paradigms like quantum information retrieval, quantum searches, *quantputers*, and their implications to encryption and information security.

During the last 20 years, quantum computing has mature and is now in the fast lane.

We already have quantum computers, quantum programming languages, and quantum pagerank algorithms. We even have quantum hackers and crackers.

So university computer science departments may want to start embracing quantum-oriented research projects and affine technologies. Same goes for private companies and marketing research companies.

So the challenge for this 2017 and upcoming years is…

**“To bit, or not to bit, that is the qubit:”**

Filed under: Algorithms, Big Data, Data Mining, Homeland Security, Mathematics, New Information Retrieval Paradigms, Quantum Computing, Quantum Information Retrieval, Quantum Searches, Statistics and Mathematics Tagged: Data Mining, Mathematics, Quantum Computing, Quantum Information Retrieval, Quantum Searches, search engines, tools ]]>

A new University of Liverpool (1) study published in Neuropsychology! (2) indicates that it may be possible to detect Alzheimer’s, Parkinson’s and other neurodegenerative disorders in artists before they are diagnosed.

The abstract (2, 3) of *What paint can tell us: A fractal analysis of neurological changes in seven artists.* and authored by Forsythe, Alex; Williams, Tamsin; Reilly, Ronan G.

Neuropsychology, Vol 31(1), Jan 2017, 1-10 states and quote:

“Can age-indexed variations in the fractal dimension of the works of artists anticipate specific cognitive deteriorations?

Method: To answer this question we analysed age-related variations in the fractal dimension of a large corpus of digital images (n = 2092) of work created by seven notable artists who experienced both normal ageing and neurodegenerative disorders.

Results: The results of our analysis showed that patterns of change in the fractal dimension of the paintings differentiated artists who suffered neurological deterioration from those of normal aging controls.

Conclusions: These findings are of importance for two reasons. Our work adds to studies that demonstrate that fractal analysis has the potential to determine the provenance of paintings. Secondly, our work suggests that may be possible to identify a-typical changes in the structure of an artist’s work; changes that may be early indicators of the onset of neurological deterioration.”

End of the quote.

References

1. https://news.liverpool.ac.uk/2016/12/29/can-paint-strokes-help-identify-alzheimers/

2. http://psycnet.apa.org/journals/neu/31/1/1/

3. http://dx.doi.org/10.1037/neu0000303

Filed under: Algorithms, Data Mining, Fractal Geometry, Mathematics, Mind Retrieval, News, Statistics and Mathematics Tagged: Algorithms, Data Mining, Fractal Analysis, Fractal Geometry, fractals, Mathematics, statistics ]]>

“The National Institute of Standards and Technology (NIST) is officially asking the public for help heading off a looming threat to information security: quantum computers, which could potentially break the encryption codes used to protect privacy in digital systems. NIST is requesting methods and strategies from the world’s cryptographers, with the deadline less than a year away.”

Read more at:

http://phys.org/news/2016-12-nist-future-proof-electronic.html#jCp

Now that Quantum Computers and Quantum Searches are at a corner near you, the implications are many: from search marketing to search apps, from social grids, to quantum PCs, from big challenges to big data, from quantum retrieval to mind retrieval: The sky is the limit. Back in 2013 we mentioned quantum searches in the context of XOR/XNOR searches.

http://www.minerazzi.com/help/xor-xnor.php

A miner on quantum searches will soon be available at http://www.minerazzi.com. In the meantime, see some useful links below:

References

- Phys.org (2016). NIST asks public to help future-proof electronic information.
- Viamontes, G. F., Markov, I. L., & Hayes, P. (2005). Is Quantum Search Practical?
- Phys.org (2005). Data structures influence speed of quantum search in unexpected ways.
- Quora (2014). How do you use the Grover quantum search algorithm to find all the solutions to some search query?
- Paparo, G. D. & Martin-Delgado, M. A. (2012). Google in a Quantum Network.
- Wang, H., Wu, J., Yang, X., Chen, P., & Yi, X. (2014). An Enhanced Quantum PageRank Algorithm Integrated with Quantum Search.
- Lu, S., Zhang, Y., & Liu, F. (2013). An efficient quantum search engine on unsorted database.
- MIT Technology Review (2011). Quantum PageRank Algorithm Outperforms Classical Version.

Filed under: Algorithms, Big Data, Data Mining, Homeland Security, Human-Computer Interaction, IR Tools, Mind Retrieval, New Information Retrieval Paradigms, Programming, Quantum Computing, Quantum Information Retrieval, Quantum Searches, Queries, social mining, Software Tagged: Algorithms, Data Mining, ir, Physics, Quantum Computing, Quantum Information Retrieval, Quantum Searches ]]>

12-23-2016 Update: BREAKING NEWS: LIGO discovery named Science’s 2016 Breakthrough of the Year.

http://phys.org/news/2016-12-ligo-discovery-science-breakthrough-year.html

As its name states, our LIGO miner finds resources relevant to the Laser Interferometer Gravitational-Wave Observatory (LIGO) project, one of the largest NSF-funded projects and that is praised by the scientific community for its discoveries, receiving a lot of attention, recognition, and prizes all over the world.

The LIGO Project allows scientists to better understand and see the Universe: i.e., to see and analyze gravitational waves due to distant objects and events, like the collision and merging of black holes. It has already proven Albert Einstein theory of gravitational waves. See these links

https://www.ligo.caltech.edu/news/ligo20160211

https://www.ligo.caltech.edu/news/ligo20160615

https://www.ligo.caltech.edu/detection

A course on the subject has been available online for a long time at

http://elmer.tapir.caltech.edu/ph237/

To learn more about LIGO, visit these links:

https://www.ligo.caltech.edu

https://www.ligo.caltech.edu/page/mission

https://www.ligo.caltech.edu/page/facts

https://www.ligo.caltech.edu/page/facilities

https://www.ligo.caltech.edu/page/timeline

https://www.ligo.caltech.edu/page/blog

https://www.ligo.caltech.edu/page/faq

https://www.ligo.caltech.edu/page/for-media

https://www.advancedligo.mit.edu

http://www.ligo.org

This is a new way of seeing the Universe. It also opens the door for new technologies at the intersection of many disciplines like noise reduction, optics, among others; hence the importance of developing this miner.

Back in 1991, the New York Times reported that experts clashed over the project. Back then Dr. J. Anthony Tyson, an astrophysicist at A.T.&T. Bell Laboratories, and at the time chairman of the Astronomy Advisory Committee of the National Science Foundation, polled astronomers as to their views about the project: “I perused a list of about 2,000 astronomers and picked 70 who seemed to me likely to have thought about LIGO,” Dr. Tyson said in an interview. “I got 60 replies, and they ran 4 to 1 against LIGO. Most of the astrophysical community seems to feel it would be very difficult to get any important information from a gravity-wave signal, even if one should be detected.”

See the full story at this link:

Those detractors were all wrong! 26 years latter, I wonder what happened with them or what they are thinking these days.

The Challenge Now:

Are you ready for LIGO and a new kind of Astronomy?

Would you like to build curated collections about LIGO?

Then try our miner.

PS

Here is a nice Mashable article on the subject

http://mashable.com/2016/02/11/einstein-gravitational-waves-detection/

discovered with the miner for the query [einstein].

Filed under: Algorithms, Data Mining, IR Tools, LIGO, New Information Retrieval Paradigms, News, Programming, Software Tagged: astronomy, Data Mining, gravitational waves, LIGO, ligo miner, Mathematics, minerazzi, miners, NSF, Physics, tools ]]>

For those that at this inning of the game are still interested in the old podesta-clinton email saga (not sure why), try this:

A search for [podesta] in the WikiMiner retrieves records relevant to Podesta emails. Once in the results page you may recursively discover new records by recrawling the results.

For instance, search for [podesta] and click the Links Tool (black rectangular chain icon) under the second result (https://www.wikileaks.com/podesta-emails) to discover new records. You will be presented with two set of records: Externals and Internals .

The Links tool now displays at the right of each new record. Click the tool again, for instance for one the Internal results, to discover more records. By repeating the process you will be recrawling and discovering over and over new records.

You may play in this way with some of the miners available at http://www.minerazzi.com

Filed under: Algorithms, Data Mining, Human-Computer Interaction, Queries, Software Tagged: curated collection, Data Mining, miner, minerazzi, podesta emails, Recrawling, tools, wikileaks miner, wikiminer ]]>

1. continuous mode: without stopping.

2. discrete mode: controlled by the user.

The first mode quickly fills a small database, but still makes the crawler to act like a vacuum cleaner, collecting all kind of garbage; i.e. links that might be irrelevant to a topic-specific database are also grabbed.

The second one is slower, but lets me decide whether to continue or stop the crawling at a given recursion level and based on topic criteria, reducing the amount of garbage gathered. Relative urls are automatically resolved into absolute ones, a bit tricky task when recursion is involved.

In addition to links, the crawler selectively extracts strings that match specific patterns like email addresses, phone numbers, zip codes, etc. Contextual keywords surrounding the patterns are also collected so these can be reciprocally mapped. The goal is the development of a service that consumes in-context pattern-specific databases–great for people searches, intelligence, and marketing. So far the project simplifies even more the building of topic miners.

Reference

http://infolab.stanford.edu/~olston/publications/crawling_survey.pdf

Filed under: Algorithms, Crawlers, Data Mining, Graduate Courses, IR Tools, Marketing Research, Programming, Search Modes, Software Tagged: Algorithms, crawler, crawling, Data Mining, tools ]]>

We used them to test yahoo.com’s IP (98.139.183.24).

Replace any instances of 98.139.183.24 with the one you want to test.

https://www.robtex.com/ip/98.139.183.24.html

http://www.robtex.com/whois/98.139.183.24.html

https://db-ip.com/98.139.183.24

http://whatismyipaddress.com/ip/98.139.183.24

https://tools.wmflabs.org/guc/index.php?user=98.139.183.24

http://www.dnsstuff.com/tools/

http://reportcard.wmflabs.org/

https://petscan.wmflabs.org/

http://tools.wmflabs.org/render-tests/catcycle-dev/catcycle.py

http://whois.arin.net/rest/ip/98.139.183.24.html

http://wq.apnic.net/apnic-bin/whois.pl?searchtext=98.139.183.24

http://www.afrinic.net/cgi-bin/whois?searchtext=98.139.183.24

http://www.ripe.net/fcgi-bin/whois?searchtext=98.139.183.24

https://rdap.lacnic.net/rdap-web/ip?key=98.139.183.24

Filed under: Data Mining, Internet Engineering, IR Tools, Marketing Research, Spam Tagged: business intelligence, intelligence, ip address, Marketing Research, spam, tracking users, wikipedia ]]>

Filed under: Curated Collections, Data Mining, IR Tools, Programming Tagged: miislita, minerazz, miners ]]>

http://www.minerazzi.com/tools/must/must.php

This is a redirection checker tool that upon url redirections reports initial and final status codes, URLs, and IP addresses.

The tool now:

1. accepts 500 urls per submission.

2. summarizes broken and active URLs.

This is one of several tools that we use in-house for re-indexing databases and cleaning up crawl results, except without url limitations.

Filed under: Algorithms, Data Mining, IR Tools, Programming, Scripts, Software Tagged: Algorithms, information retrieval, ir, minerazzi, miners, tools ]]>

It is available at http://www.minerazzi.com/shopper/

The miner now features an SPP-powered news service with consumer alerts, recalls, and fraud reports

Use it to find coupon deals, price comparisons, holidays offers, consumer reports, and more.

Filed under: Curated Collections, Data Mining, Marketing Research, Miscellaneous, News, social pulse parser Tagged: consumer reports, coupons, holiday season, miner, news, shopper, shopping, special deals, special offers ]]>

http://www.minerazzi.com/e-paper

More than appropriate miner on electronic paper and ink technologies, now that the news is out about Microsoft’s sticky notes using e-ink powered by ambient light.

E-Paper and E-Ink technologies are set to simplify our life: http://www.psfk.com/2016/10/microsofts-e-paper-note-runs-on-ambient-light.html

https://www.technologyreview.com/s/602710/this-e-ink-post-it-never-needs-to-be-charged/

Filed under: Algorithms, Curated Collections, Data Mining, IR Tools, News, Programming, Technology Inventions Tagged: Algorithms, Curated Collections, Data Mining, miner, minerazzi ]]>

http://www.minerazzi.com/mobile

Find apps software and vendors for mobile devices, smartphone technology, and more. Search by vendors, makers, models, or keywords. Submit web pages relevant to these topics.

Includes an RSS News section powered by SPP to help you find industry-related news, releases, events, and more so far from AT&T, Samsung, Sprint, and Verizon. Feel free to submit for inclusion a relevant RSS url

Filed under: Curated Collections, Data Mining, ir, IR Tools, News, Programming, Queries, Scripts, social pulse parser, Software Tagged: Data Mining, miner, minerazzi, social pulse parser ]]>

To download the Excel file, access the most recent update of the tutorial, available at

http://www.minerazzi.com/tutorials/quantile-quantile-tutorial.pdf

We also removed few extra “)” typos that were undetected in previous copies.

Have a great Q-Q day!

Filed under: Algorithms, Data Mining, IR Tools, Mathematics, Programming, Statistics and Mathematics Tagged: Data Mining, Mathematics, statistics, tutorials ]]>

http://www.minerazzi.com/tools/text-streamer/streamer.php

Streamline text by removing non-printable or encoded characters and multiple spaces.

The tool converts non-printable characters, including tabs, returns, newliners, and multiple spaces into single spaces. User can opt to remove all encodes. These are characters encoded in %, decimal, and hexadecimal notation.

To use the tool, just enter your input text and submit form. To remove all encodes, check the form checkbox. Click the output text to select it. Copy/paste it as usually you would.

It comes handy for users that need to copy/paste streamlined text (plain text) from one file type to another or post it through html forms residing in blogs, discussion forums, and social network sites, or any site for that matter.

Filed under: Algorithms, Data Conversion, Data Mining, IR Tools, Programming, Scripts, Software Tagged: Algorithms, text streamer, tools ]]>

http://www.minerazzi.com/tutorials/bm25f-model-tutorial.pdf

Active links were also added to the References section.

Enjoy it.

Filed under: Algorithms, Best Match Models (BM), Data Mining, ir, IR Tutorials, New Information Retrieval Paradigms, News, Queries Tagged: Algorithms, Data Mining, tutorials ]]>

http://www.minerazzi.com/nobel-prize/

Use it to find resources relevant to laureates of the Nobel Prize. Search by laureates, country, discipline, or field. Find Nobel Prize Laureates in Chemistry, Physics, and other fields.

Filed under: chemistry, Data Mining, ir, IR Tools, Mathematics, News, Programming, Scripts, Software, Statistics and Mathematics, Theses Tagged: Algorithms, chemistry, Data Mining, Mathematics, Physics, statistics, tools ]]>

http://www.minerazzi.com/tools/matrix-inverter/gauss-jordan.php

and mentioned in the post

https://irthoughts.wordpress.com/2016/10/05/matrix-inverter-a-matrix-inversion-tool/

The following information was found online (Quora, 2013, StackExchange, 2013a; 2013b).

Let Ʃ be a covariance matrix and Ʃ

^{-1}an inverse covariance matrix, commonly referred to as the precision matrix.With Ʃ, one observes the unconditional correlation between a variable i, to a variable j by reading off the (i,j)-th index.

It may be the case that the two variables are correlated, but do not directly depend on each other, and another variable k explains their correlation. By computing Ʃ

^{-1}we can examine if the variables are partially correlated and conditionally independent.Ʃ

^{-1}displays information about the partial correlations of variables. A partial correlation describes the correlation between variable i and j, once you condition on all other variables. If i and j are conditionally independent then the (i,j)-th element of Ʃ^{-1}will equal zero. If the data follows a multivariate normal then the converse is true, a zero element implies conditional independence.In general, Ʃ

^{-1}is a measure of how tightly clustered the variables are around the mean (diagonal elements) and the extend to which they do not co-vary with the other variables (non-diagonal elements). The higher the diagonal elements, the tighter the variables are clustered around the mean.

So far I found that to be, in my opinion, the simplest explanation on the subject. So there you have a good application for our Matrix Inverter tool.

References

- Quora (2013). What is the inverse covariance matrix?.
- StackExchange (2013a). How to interpret an inverse covariance or precision matrix?.
- StackExchange (2013b). What does the inverse of covariance matrix say about data? (Intuitively).

Filed under: calculators, Data Mining, IR Tools, PCA, Programming, Scripts, Software, Statistics and Mathematics Tagged: Algorithms, Data Mining, Mathematics, statistics, tools ]]>

http://www.minerazzi.com/tools/matrix-inverter/gauss-jordan.php

The tool inverts a square matrix using Gauss-Jordan Elimination.

A matrix filled with zeroes is returned if the input matrix is non-invertible. This is used as a crude signal.

A non-invertible square matrix, also called singular or degenerate, is one whose determinant is zero.

The tool can be used to double check calculations of small matrices or as a demo resource.

Have a nice invertible day.

Filed under: Algorithms, calculators, Data Conversion, Data Mining, Mathematics, Programming, Scripts, Software, Statistics and Mathematics Tagged: Algorithms, Mathematics, statistics, tools ]]>