• About IR Thoughts
  • Minerazzi Tools
  • Minerazzi Tutorials

IR Thoughts

~ Thoughts on Information Retrieval, Search Engines. Data Mining, and Science & Engineering

IR Thoughts

Category Archives: RSS/Atom Feeds

W3C Miner

13 Wednesday Feb 2019

Posted by egarcia in Curated Collections, Data Mining, IR Tools, miner, minerazzi, News, RSS/Atom Feeds, Software, Web Mining

≈ Leave a comment

Tags

Curated Collections, miner, minerazzi, W3C, W3C Miner

W3C Miner. http://minerazzi.com/w3c/

W3C public resources, news, standards, work groups, and more.

Use its news channel to easily access from a single place several rss news feeds relevant to World Wide Web Consortium.

Recrawl search results and build your own curated collection of resources.

Advertisements

ORNL Miner

07 Thursday Feb 2019

Posted by egarcia in Data Mining, Homeland Security, National Laboratories, Research Centers, RSS/Atom Feeds, Software, Technology Inventions, Theoretical Physics

≈ Leave a comment

Tags

Cutting-Edge Research, National Laboratories, ORNL, Technology Inventions

ORNL Miner is now available at

http://www.minerazzi.com/ornl

Find ORNL public resources, news, research, people, and more. Access cutting-edge research and technology inventions.

Use its news channel to easily access from a single place several rss news feeds relevant to Oak Ridge National Laboratory. 

Recrawl search results and build your own curated collection of resources.

Google’s Data Sets Search Engine

05 Wednesday Sep 2018

Posted by egarcia in Algorithms, Data Conversion, Data Mining, Feed Tools, IR Tools, miner, minerazzi, Programming, Public Databases, Public Records, RSS/Atom Feeds

≈ Leave a comment

Tags

Data Mining, data sets, open data sets, rss mining

This is a great tool from Google
https://www.blog.google/products/search/making-it-easier-discover-datasets/

I will try to see how the feed flattener can benefit from it.
http://www.minerazzi.com/tools/flattener/feed-flattener.php

Document tree flattening as an exploration technique for data mining .xml files (sitemaps, feeds, inventories, raw data, etc)

25 Monday Jun 2018

Posted by egarcia in Crawlers, Data Mining, Data Structures, Feed Tools, information retrieval, IR Tools, minerazzi, Programming, RSS/Atom Feeds, Sitemaps, Software, URLs Mining, Web Mining

≈ Leave a comment

Tags

Data Mining, Feeds, Flatteners, minerazzi, Sitemaps, tools, URLs Extraction, web feeds, XML Mining

Two of our tools, Web Feed Flattener and Feed URLs Extractor, were updated and now accept files with the .xml extension so we changed their names to indicate this. These tools are available at

http://www.minerazzi.com/tools/flattener/feed-flattener.php

http://www.minerazzi.com/tools/feed-urls/extractor.php

These updates take the tools to a whole new level. Now you can flatten the tree structure of files like sitemaps.xml and similar files and extract URLs. Just submit a  target web address and you are good to go.

I know there are tools out there that can scrape .xml files in order to extract specific pieces of data like URLs, but found them too cumbersome. A major drawback of said design alternatives is that frequently one must know in advance how the document tree was constructed, with all of its tags and nuances, before coding a tool. To top off, if the author of the file changes or edits tags, probably the tool won’t work as expected.

Our approach is different and very flexible. The key here is the flattening of the document tree structure embedded in XML files without even having to know how it was designed or edited. Document tree flattening will unveil this information before you can say: “Give me some soup!”

Of course, we assume that the document tree has no orphan or broken tags (and better, pass validation) which is something to be expected from trusted sources. If it is not valid, well, there are ways of fixing it or ignore the offenders.

With the proposed technique, we can mine all sort of .xml files and build customized tools on top of the flattened results, like derivative tools for mining sitemaps, inventories, raw data, recipes, etc… No need to know in advance anything about the document tree, resource to additional scripting technologies, software, or reinvent the wheel.

Right now we can mine sitemaps all over the Web, including sitemaps hosted at Google, W3C, company sites, etc, and then recrawl the output to grow a microindex. See “Suggested Exercises” sections of the tools for interesting examples. This is a value-added approach for our Maps2Miners ongoing project.

Considering that there are government agencies and organizations facilitating data in .xml format for developers to mine, flattening .xml files and build on top of these is one of those “ah-ha!” ideas.

 

RSS/Atom Feeds Miner

28 Monday May 2018

Posted by egarcia in Data Mining, Feed Tools, information retrieval, IR Tools, miner, minerazzi, News, RSS/Atom Feeds, social mining

≈ Leave a comment

Tags

Atom, Feeds, minerazzi, RSS, RSS/Atom Feeds, web feeds

Having problems finding research focused news feeds? Try our RSS/Atom Feeds miner (http://www.minerazzi.com/feeds).

Use it to find search engines, directories, and sites listing topic-specific news feeds. Then recrawl the miner search results and build your own list or collection of news feeds. Supports the popular RSS and Atom formats, among others.

Use the results from this miner in combination with other tools we developed for data mining URLs, or if you prefer submit your own page listing news feeds and improve the web visibility of your news.

This miner comes handy, particularly now that Google News has disabled its RSS subscription features, breaking tons of feeds and, in the process, affecting their presence on the web. (https://www.seroundtable.com/google-news-rss-feed-gone-25795.html).

February 2019
M T W T F S S
« Jan    
 123
45678910
11121314151617
18192021222324
25262728  

Favorite Sites

  • Minerazzi.com

Pages

  • About IR Thoughts
  • Minerazzi Tools
  • Minerazzi Tutorials

Categories

  • 4D Printing
  • AIRWeb Course
  • Algorithms
  • Amazon Alexa
  • Android
  • Arithmetic Geometry
  • Best Match Models (BM)
  • Big Data
  • BioDesign
  • bioinformatics
  • Blogroll
  • calculators
  • Cancer
  • Cancer Research
  • Causality & Determinism
  • Chaos
  • chemical mining
  • Chemist Biographies
  • chemistry
  • Chemometrics
  • Clinical Trials
  • Cloud Computing
  • Conferences
  • Correlation Coefficients
  • Cortana
  • Crawlers
  • Curated Collections
  • Data Conversion
  • Data Mining
  • Data Structures
  • Deep Neural Networks
  • Docker
  • Dynamics
  • Favorite Sites
  • Feed Tools
  • Fisher Transformations
  • Fractal Geometry
  • Fractal Patterns
  • Graduate Courses
  • Hacking
  • Homeland Security
  • Human-Computer Interaction
  • Image Compression
  • information retrieval
  • Internet Engineering
  • inverted index
  • ir
  • IR Quizzes
  • IR Tools
  • IR Tutorials
  • Kubernetes
  • Latent Semantic Indexing
  • Legacy Posts
  • LIGO
  • Machine Learning
  • Marketing Research
  • Mathematics
  • Medical Cannabis
  • meta-analysis
  • Mind Retrieval
  • miner
  • minerazzi
  • Miscellaneous
  • National Laboratories
  • New Information Retrieval Paradigms
  • News
  • Newsletters
  • Nonlinear Dynamics
  • One-to-Many (O2M)
  • Open Source Projects
  • Patents
  • PCA
  • People Searches
  • Perfectoid Spaces
  • Physiology
  • Poems
  • political networks
  • Predatory Conferences
  • Predatory Journals
  • Programming
  • Public Databases
  • Public Records
  • Quack Science
  • Quantum Computing
  • Quantum Information Retrieval
  • Quantum Searches
  • Quantum Theory
  • Queries
  • Research Centers
  • RSS/Atom Feeds
  • Scripts
  • Search Engines Architecture Course
  • Search Modes
  • self-weighting
  • SEO Myths
  • Sitemaps
  • social mining
  • social pulse parser
  • Software
  • Spam
  • Statistics and Mathematics
  • SVD
  • Technology Inventions
  • Theoretical Physics
  • Theses
  • twitter
  • URLs Mining
  • Vector Space Models
  • Voice Assistants
  • Web Mining
  • Web Mining Course
  • Web Security
  • Xamarin

Recent Posts

  • W3C Miner
  • ORNL Miner
  • Carcinomas Miner
  • Bond Order Calculator Tool
  • Applications of Binary Fractals to Long Genetic Sequences via a Kronecker Family of Genetic Matrices
  • On Matrix Determinants, Poems, and Symmetry
  • On the Non-Additivity of Correlation Coefficients Part 3: The Bias & Nature of Correlation Coefficients
  • Hybrid Similarity Search (HSS) Algorithm for Chemistry Searching
  • C.R. Rao: A Giant & Living Legend of Statistics Among Us
  • On Mind Retrieval: BrainNet
  • On Men and Ideas: Fisher vs. Pearson
  • Correlation and Regression Theory from 1930-1957
  • Predatory Journals Miner
  • This one is predatory. That one isn’t. Really? Are all journals predatory?
  • Regression & Correlation Calculator: Updates and Improvements

Archives

  • February 2019
  • January 2019
  • December 2018
  • November 2018
  • October 2018
  • September 2018
  • August 2018
  • July 2018
  • June 2018
  • May 2018
  • April 2018
  • March 2018
  • February 2018
  • January 2018
  • September 2017
  • August 2017
  • July 2017
  • June 2017
  • May 2017
  • April 2017
  • March 2017
  • February 2017
  • January 2017
  • December 2016
  • November 2016
  • October 2016
  • September 2016
  • August 2016
  • July 2016
  • June 2016
  • May 2016
  • April 2016
  • March 2016
  • February 2016
  • January 2016
  • December 2015
  • November 2015
  • October 2015
  • September 2015
  • August 2015
  • July 2015
  • June 2015
  • May 2015
  • April 2015
  • March 2015
  • February 2015
  • January 2015
  • December 2014
  • November 2014
  • October 2014
  • September 2014
  • August 2014
  • July 2014
  • June 2014
  • May 2014
  • April 2014
  • March 2014
  • January 2014
  • December 2013
  • November 2013
  • October 2013
  • September 2013
  • August 2013
  • July 2013
  • May 2013
  • April 2013
  • March 2013
  • February 2013
  • January 2013
  • December 2012
  • November 2012
  • October 2012
  • September 2012
  • August 2012
  • July 2012
  • June 2012
  • May 2012
  • April 2012
  • March 2012
  • February 2012
  • January 2012
  • December 2011
  • November 2011
  • October 2011
  • September 2011
  • August 2011
  • July 2011
  • June 2011
  • May 2011
  • April 2011
  • February 2011
  • January 2011
  • December 2010
  • November 2010
  • October 2010
  • September 2010
  • August 2010
  • July 2010
  • June 2010
  • May 2010
  • April 2010
  • March 2010
  • February 2010
  • January 2010
  • December 2009
  • November 2009
  • October 2009
  • September 2009
  • August 2009
  • July 2009
  • June 2009
  • May 2009
  • April 2009
  • March 2009
  • February 2009
  • January 2009
  • December 2008
  • November 2008
  • October 2008
  • September 2008
  • August 2008
  • July 2008
  • June 2008
  • May 2008
  • April 2008
  • March 2008
  • February 2008
  • January 2008
  • December 2007
  • November 2007
  • October 2007
  • September 2007
  • August 2007
  • July 2007
  • June 2007
  • May 2007
  • April 2007

Algorithms calculators chemical mining Conferences Curated Collections Data Mining Hacking Homeland Security Human-Computer Interaction IR Tools IR Tutorials Latent Semantic Indexing Machine Learning Marketing Research Mathematics minerazzi Miscellaneous New Information Retrieval Paradigms News Newsletters Programming Queries Scripts Search Engines Architecture Course SEO Myths Software Spam Statistics and Mathematics Vector Space Models Web Mining Course
Advertisements

Blog at WordPress.com.

Privacy & Cookies: This site uses cookies. By continuing to use this website, you agree to their use.
To find out more, including how to control cookies, see here: Cookie Policy