A Simple Example of Phonetic Similarity vs. Text Similarity

Tags

levenshtein distance, levenshtein similarity, minerazzi tools, phonetic similarity, tools

The followings sound the same so their phonetic similarity is 1.

(a) r u?
(b) ar u?
(c) are u?
(d) r you?
(e) ar you?
(f) are you?

However, the Levenshtein Distance (LD) and Levenshtein Similarity (LS) of (a) with the other strings differ:

LD(a, b) = 1; LS(a, b) = 0.5
LD(a, c) = 2; LS(a, c) = 0.33
LD(a, d) = 2; LS(a, d) = 0.33
LD(a, e) = 3; LS((a, e) = 0.25
LD(a, f) = 4; LS(a, f) = 0.2

Can you find LD and LS results for other possible combinations?

For i = j, LD(i, j) = 0 and LS(i, j) = 1 so you may want to ignore this case.

Note: LD and LS results were computed with our tool at http://www.minerazzi.com/tools/levenshtein/levenshtein-distance-calculator.php

References
http://www.minerazzi.com/tutorials/levenshtein-distance-tutorial.pdf
http://www.minerazzi.com/tutorials/distance-similarity-tutorial.pdf

Advertisements

Zillman’s 2019 Directory of Directories

Tags

directories, Directory of Directories, miner, minerazzi

Zillman’s 2019 Directory of Directories is a handy resource for those interested in finding specialized gateways to the Web.

Glad to know that this summer Minerazzi was included in the Academic/Education section.

(http://columns.virtualprivatelibrary.net/2019_Directory_Of_Directories_June19_column.pdf)

See also http://www.2019directoryofdirectories.com/

Since 2018 we are listed in the Bot and Intelligent Agent Resources category and few other sections, but did not realize that in that one too.

(http://www.zillman.us/minerazzi-your-search-and-mine)

Cool!

More on Perfectoid Spaces

Tags

espacios perfectoides, fractals, Mathematics, perfectoid miner, Perfectoid Spaces

Back in 08-12-2018, we blogged on Peter Scholze’s 2011 thesis on Perfectoid Spaces, one of the hottest topics that is taking Mathematics by storm (https://irthoughts.wordpress.com/2018/08/12/peter-scholze-and-perfectoid-spaces-a-math-genius-among-us/).

An introductory discussion on the topic is available at https://mathoverflow.net/questions/65729/what-are-perfectoid-spaces?rq=1

Possible connections and applications are mentioned at https://irthoughts.wordpress.com/2018/08/23/perfectoid-spaces-arithmetic-geometry-and-quantum-theory/.

The Perfectoid Spaces miner is available since 08-16-2018 (https://irthoughts.wordpress.com/2018/08/16/perfectoid-spaces-miner/).

Resources in spanish were added to the miner yesterday 08-26-2019, about a year later. Better late than never.

Extracting Topic-Specific Wikipedia Links

Tags

miner, minerazzi, wiki links, wikipedia

Below is an illustrative example of a miner for extracting specific links from Wikipedia.

http://www.miislita.com/wikilinks

This miner provides entry points to Wikipedia links relevant to Puerto Rico.

Recrawling a search result discovers new topic-specific links, allowing for further exploration and mining.

Programming Languages Miner

Tags

Python, Programming, PHP, JavaScript, programming languages, computer programming, C++, PERL

Programming Languages Miner (http://www.minerazzi.com/programming-languages/)

Find lists of programming languages, tutorials, and related resources.

Recrawl results to build your own curated collection.

Its news channels include direct access to blog feeds of some of the most popular languages, like C++, JavaScript, PERL, PHP, and PYTHON.

New IANA Miners

Tags

iana, Internet Standards, minerazzi, miners

Here is a series of IANA miners, available now at Minerazzi.com.

IANA Drafts: http://www.minerazzi.com/iana-drafts/

IANA Domains: http://www.minerazzi.com/iana-domains/

IANA Assignments: http://www.minerazzi.com/iana-assignments/

IANA Reports: http://www.minerazzi.com/iana-reports/

Resources relevant to IANA members can also be found through these.

For instance, search for [email]. In the search results page, click the emails tool icon at the top of the results…

How to Lie with Bar Graphs: A Tool for Agents of Misinformation and Fake News

Tags

agents of misinformation, fake news, paid polls, politicians, propaganda, seos

In the article, The Use and Abuse of Logarithmic Axes, Harvey J. Motulsky explained how bar graphs can be used for propaganda and misleading others. The article is at

https://s3.amazonaws.com/cdn.graphpad.com/faq/1910/file/1487logaxes.pdf

He shows the following graph and explains, and quote

Since zero can’t be shown on a log axis, the choice of a starting place is arbitrary. Accordingly, the relative height of two bars is not directly related to their relative values. The graphs below show the same data as the graph above, but with three different choices for where the axis begins. This arbitrary choice influences the relative height of the two bars, amplified in the graph on the left and minimized in the graph on the right.

If the goal is to create propaganda, a bar graph using a logarithmic axis is a great tool, as it lets you either exaggerate differences between groups or minimize them. All you have to do is carefully choose the range of your axis. Don’t create bar graphs using a logarithmic axis if your goal is to honestly show the data.

So now you know. Never trust anyone that uses such tricks (politicians, pollsters, marketers, sales department staff, seos, and the likes). This one is also a great tool for fake news masters.

A Unicode Mnemonic for Vowels with Diacritics

Tags

Mnemonic, Unicode, Unicode Mnemonic, Vowels with Diacritics

A mnemonic for easily recalling the unicode entities of vowels with grave accents, acute accents, and circumflexes is available at

http://www.minerazzi.com/tutorials/unicode-mnemonic.pdf

We developed the mnemonic after some work on mining the unicode system. It might come handy for those that need to encode said vowels without looking at reference books, tables,… It is also a great example for those teaching mnemonics.

For those interested, we have a tool for exploring unicode entities at

http://www.minerazzi.com/tools/unicoder/unicoder.php

Update: I found a typo in the “o” lower case circumflex and also an error in the 194 entity. Fixed both. Sorry.

Dataset for Sequential Sentence Classification

Tags

Annotation, Classification, information retrieval, ir, Power Transformations

A research group at CSIE department of National Taiwan University (NTU), supervised by Prof. Shou-De Lin, is currently working on releasing a new dataset for sequential sentence classification and type classification for paper abstracts on arXiv.

 

This annotation and classification project is available at https://mslab.csie.ntu.edu.tw/~labeler/abstract.php

 

Annotations are made by article authors and upon being invited/contacted by the research group. I’m happy to see they included in the dataset my “Local Term Weights Models from Power Transformations | Development of BM25IR” paper (https://arxiv.org/ftp/arxiv/papers/1608/1608.01573.pdf).

PHP Implementation of Text Mining in Digital Libraries using OKAPI BM25 Model

Tags

Apache, BM25, BM25 Tutorial, BM25F, BM25F Tutorial, minerazzi, open source, PHP

Here is a nice article from International Journal of Computer Applications Technology and Research, https://ijcat.com/archieve/volume7/issue10/ijcatr07101003.pdf, on text mining digital libraries with OKAPI BM25. The authors successfully used a PHP implementation (XAMPP bundle).

Happy to see in reference 22 of the article that they cited one of my tutorials, on the BM25F implementation. This one is available at http://www.minerazzi.com/tutorials/bm25f-model-tutorial.pdf (and through researchgate.net).

The authors generously included some PHP snippets so it should be easy for those that can code to follow up.