PHP Implementation of Text Mining in Digital Libraries using OKAPI BM25 Model

Tags

Apache, BM25, BM25 Tutorial, BM25F, BM25F Tutorial, minerazzi, open source, PHP

Here is a nice article from International Journal of Computer Applications Technology and Research, https://ijcat.com/archieve/volume7/issue10/ijcatr07101003.pdf, on text mining digital libraries with OKAPI BM25. The authors successfully used a PHP implementation (XAMPP bundle).

Happy to see in reference 22 of the article that they cited one of my tutorials, on the BM25F implementation. This one is available at http://www.minerazzi.com/tutorials/bm25f-model-tutorial.pdf (and through researchgate.net).

The authors generously included some PHP snippets so it should be easy for those that can code to follow up.

 

Advertisements

BM25F Model Tutorial

Tags

Algorithms, Data Mining, tutorials

We have restored, expanded, and updated our tutorial on the BM25 Extension to Multiple Weighted Fields Model, best known as BM25F. It is now available at

http://www.minerazzi.com/tutorials/bm25f-model-tutorial.pdf

Active links were also added to the References section.

Enjoy it.

OKAPI BM25 Tutorial

Tags

Algorithms, ir, minerazzi, tutorials

We have restored, refined, and updated this tutorial and added some historical background.

Abstract

This is a light tutorial on OKAPI BM25, a Best Match model where local weights are computed as parameterized frequencies and global weights as RSJ weights. Local weights are based on a 2-Poison model and the verbosity and scope hypotheses and global weights on the Robertson-Spärck-Jones Probabilistic Model.

Introduction

In the early 80s Gillian Venner, Nathalie Mitev, and Stephen Walker (1985, 1987) conducted research work that led to the design and evaluation of online public access catalogs (OPACs) at Polytechnic of Central London (PCL).

The project initial phases spanned from November 1982 to May 1985. The prototype was named OKAPI (Online Keyword Access to Public Information). As Mitev (1985) wrote:

“Designing an online public access catalogue [OPAC]: Okapi, a catalogue on a local area network [LAN] is the final report of a two-year research  project ”Microprocessor networking in libraries” which was funded by the British Library and the Department of Trade and Industry, and based at the Polytechnic of Central London.”

“The aim was to produce an OPAC on a LAN, that would be readily usable without training or experience, without sacrificing effectiveness or being tedious for experienced users.”

“The result was a functioning prototype OPAC called Okapi, which has a number of distinctive features: use is eased by coloured keys and a lack of jargon; the system uses search decision trees to select a suitable action at each stage of a search, and it performs automatic Boolean and hyper-Boolean functions where appropriate. The OPAC was installed and evaluated in one of the Polytechnic site libraries.”

Want more? Read the tutorial at

http://www.minerazzi.com/tutorials/okapi-bm25-model.pdf