Mining SEO and Internet Marketing Competitor Sites

Are you an SEO or part of a search marketing company? Want to compare SEO competitors or know how search marketing company home pages were designed? Which CSS style rules and colors were used for their look and feel or which JavaScript “gems” were used? Or perhaps you need to check for possible misconfigurations or robot.txt exploits, keyword distributions, etc…? Or maybe you just want to quickly check if they have added new or fresh content?

Use the Internet Marketing Search Engine (http://www.minerazzi.com/seominer). Its index, now in the order of thousands, is a curated collection of SEO and search marketing resources out there. Feel free to submit a URL if it is not already indexed.

The Data Structures and Algorithms Collection

The Data Structures and Algorithms Collection is a new miner available at http://www.minerazzi.com/dsac . This is an example of using Minerazzi to turn a pre-existent link directory or already curated collection of links into a data mining repository. We have built this miner from NIST’s Dictionary of Algorithms and Data Structures (DADS) repository (http://www.nist.gov).

Although it is now limited to NIST’s DADS, in the next days we will be adding URLs to this miner from many other bigger repositories on data structures and algorithms available across the Web. In this way students, teachers, and researchers can have in one place all these valuable programming resources. It is a great tool for graduate students and the general public interested in computer programming.

The Job Searches and Career Resources Collection

Tags

The Job Searches and Career Resources Collection is a new miner available at http://www.minerazzi.com

Use this miner to search job databases and the best career-oriented sites. Find job lists and career advice resources. Check prospective employers and recruiting companies. Submit job offers.

As usual, you can use it to mine the results from a search or compare how the resources listed were designed.

Staying Away from Spamked Domains

I call spamked domains those parked domains that spam your eyes and search experience across the web.

How pervasive this problem is and how do you fight the war against web spam?

In a recent check of URLs using MUST (1) we found that the domain http://www.metalib.com is a spamked domain that resolves to the IP 69.172.201.208

Checking this IP with MHM (2), we found that 50,673 domains share the same IP. Random sampling these reveals that these are either spamked domains or not founds.

So how do you avoid them all? Good question.

A partial solution consists in building a collection of spamked IPs and URLs which can then be queried with a candidate IP/URL. Using a dedicated crawler to recrawl these records should help you update the collection as a result of any change in IPs/URLs, something you would expect from the usual suspects.

This might not be a bullet-proof solution as may result in some inocent victims having a ‘bad company’ (host provider) and being in the ‘wrong place’ (server) at the ‘wrong time’ (crawl timestamp), but hey inocent casualties are part of any war.

Update: Our MUST and MHM tools found that the URL http://ww1.theslammer.com/ resolves to IP 72.52.4.90 and is also shared by 57,451 domains.  Also the URL http://www.bestedeal.com resolves to IP 208.73.211.250 which is shared by 10,742 domains.

Thus, with few initial IPs, we were able to identify more than 100,000 cases. Building a collection of spamked domains is not that hard to do after all.

References

1. MUST (http://www.minerazzi.com/tools/must/must.php).

2. MHM (http://www.minerazzi.com/tools/mhm/mhm.php).

Chikungunya Virus Resources

A new miner about the nasty Chikungunya Virus is now available at http://www.minerazzi.com/chikun/.
Conduct research on this hot topic. Search the scientific literature for Chikungunya, a virus disease transmitted by mosquitoes. Read newspaper articles and research on CHIKV from CDC, NIH, WHO, and other sources.
This is a must needed miner built with Minerazzi. Great for students, researchers, and those that were hit by this nasty virus,
Researchers and students: Have a field day.

Finding Financial Data and News Stories with Minerazzi

Now you can find financial and news stories with these tools:

(1) The Financial Search Engine – This is a public miner for searching and mining financial web sites. It is available at http://www.minerazzi.com/mystocks

(2) The News Center – This is a free service that eventually will be available to all miners built with the platform. So far the service returns stock market summaries and news relevant to a miner. A user can submit multiple ticker symbols or use the default symbols. News sources are randomly displayed when the page is refreshed.