Removing special characters

You can now remove special characters that Hrefer and other scrapers can’t convert. This is covered on page 8 of the manual:

Filter Out Characters & Treat Pipes as Separator (Text Snippet Settings Cont.)
Some search engine scrapers like Hrefer replace some characters before scraping. For example you may prepare a footprint list that contain this character:


This is will replaced by some scraping programs, and it means your footprint could be broken, and not return the correct results.

We have done a lot of research when it comes to search engine operators. We found that if you remove such characters, it will still return the same results in search engines like Google.

So, if your footprint is:

“General » CMS settings”

Using the following will return the same results:

“General CMS settings”

This characters are ignored, so we should exclude them, so our search engine scraper works correctly, and we get the correct results. Add characters you want to remove to the field in the Text Snippet Scraper Settings menu.

Treat Pipes as Separator
Pipes, or the “vertical bar” ( | ) appear quite often in CMS footprints. These are not effective inside search engine query strings (footprints you use to search with). If you are only interested in finding link targets then use this option to separate the parts of any snippet that contains pipes. These parts will be counted alongside all the other snippets found.