Academic Signature in the search engines


This  report, written on june 26, 2015, gives information on the visibility of the original homepage of Academic Signature (https://www.fh-wedel.de/~an/crypto/Academic_signature_eng.html) in various search engines. I wrote it after a sudden deletion of the webpage from US based google searches.
In may 2017 the latest update of my observations occurred.

 

Academic Signature and google:

Google has generally treated academic signature well. From the very beginning, it slowly climbed in page rank. Before the summer of Edward Snowden in 2013, it ranked top only in the neutral search "open source elliptic curve digital signatures".

After the snowden exposures in 2013 it also ranked on the first result pages for google:"open source public key cryptography" or "open source elliptic curve cryptography". This has been true for searches done from european or north american IP-adresses. The googlebot visits academic signature frequently and google has the page and previous versions of it cached.

Acadmic Signature in Google via disconnect.me:
Disconnect.me is a search engine interface, that claims to enhance privacy and allows to select the country of origin for google searches. Info from disconnect.me tells all their servers are located in the US and the UK.

Recently after a five eyes critical posting  in a news forum(june 24, 2015) and referring to a google search "open source elliptic curve cryptography", within hours it practically disappeared from US-based google search results. At least https://search.disconnect.me/ claims that, direct access via an US tor exit node on June 30 2015 showed academic signature ranked 2 on google-US. So it is likely, that google-US is indeed clean and that only "disconnect.me" was manipulated.

At any rate Academic Signature remained ranked among the top 4 in e.g. German, French or even Canadian based searches google:"open source elliptic curve cryptography" even via  https://search.disconnect.me.

This specific "censorship" in the US had happened before already. At that time, the google page rank had recovered within some days.

 

Academic Signature and Microsoft's Bing:

The relationship is simple. Bing had Academic Signature practically blacklisted in 2013. As far as I have tried, this is true for searches  from all parts of the world alike. Before 2013 - with a focus on digital signatures and before mentioning Edward Snowden - it had been highly visible in Bing. Since then you'll never find it with an unspecific search about open source public key crypto, elliptic curve cryptography or the like.

If you force Bing to find it with a named search(search: academic signature cryptography), Bing claims the web page had forbidden to Bing to index its content. This is not true.
I suspect the name Snowden is to Bing like a crucifix to vampires ;-) Their hard disks decay to dust if you try to store the name Snowden on them.

Recently the bing crawler started visiting academic signature's homepage several times a week without any effect on the page rank. Yahoo, owned by microsoft, even accesses project source code or binaries almost on a daily basis, yet Academic Signature's university homepage is kept out of search results.
Update 2016 till today(May 2017), Bing listings of the page changed: Academic Signature is now prominently and consistently listed. The visits by Microsoft bots occur frequently but irregularly many times a day. Despite being labeled as visits by Bots, some visits seem to adhere to human like patterns( e.g. access repeats by the same IP after some minutes or hours).

 

Academic Signature and Yandex:
Before about 2013 you could find Academic Signature easily in Yandex. "open source elliptic curve digital signatures" yielded a hit on the first page. Shortly thereafter, it had been blacklisted. After about 2014 the visibility slowly improved from extremely poor to increasingly better visibility.

As of now (June 2015) on a europe based search (Yandex: "open source elliptic curve cryptography") performed at the time of writing it ranked 15, on the second page of results. I'd like that higher, but this seems quite normal and neither suppressed nor boosted. The yandex spider visits the homepage about 2-3 times a week.
As of may 2017 the frequency of visits by the yandex spider increased a little and page rank climbed to 11, now on page 1. No substantial change of treatment occurred.

 

Academic Signature and Baidu:

Initially Baidu did not index academic signatures homepage. In the last couple of years, it made it into Baidu's index and the page rank slowly increased. As of today Baidu: "open source elliptic curve cryptography" yields rank 8 on the first result page of Baidu(searched from a german ip-address).

The Baidu crawler visits academic signature home about once a week. As of may 2017 academic signature ranked #1 on baidu when baidu was accessed via TOR(US-TOR exit node).
Indeed Academic Signature downloads to china have increased to about 1-2 a day. I have seen a substantial recent increase in visits by asian IP adresses(mostly China, Malaysia and India, a few also to South Korea and Taiwan).

 

Academic Signature and Duckduckgo:

It used to be accessible from  duckduckgo, albeit with a very low rank. As of now the assault found in Google/US coinciding with my critical post seems to also fully apply to duckduckgo. Academic Signature vanished from search results for DuckDuckGo:"open source elliptic curve cryptography".
So I speculate that duckduckgo may have at least as much anticipatory obedience towards government requests as Microsofts Bing. This is in stark contrast with duckduckgo's image. (Of course you are free to assume I just don't like duckduckgo because it considers my site irrelevant ;-)...
As of may 2017 duckduckgo ranks Academic Signature prominetly. On may 22nd, accessed via a canadian TOR-Exit, duckduckgo ranked Academic Signature #5  on "open source elliptic curve cryptography".

 

Academic Signature and Fastbot:

Fastbot is a german search engine maintaining an own independent index. As of today Academic Signature ranks top on fastbot:"open source elliptic curve cryptography" and 14 on "open source public key cryptography". This is not as good as the friendly european google listing, but I perceive it as fair and reasonable.
Update: On September 3rd, I noticed that academic signature was removed from fastbot's index. They don't even show GnuPG on "open source public key cryptography" any more but present numerous completely useless links.
Fastbot turned into crap. During the last two years the german search engine Fastbot listed just junk pages in response to "open source elliptic curve cryptography".

 

Academic Signature and meta search engines:

Meta search engines usually yield an averaged rank regarding the included primary engines. As verified today(june 26 2015), there are unusual results for "open source elliptic curve cryptography" on the german meta engine www.metager.de .

In the default setting, metager: "open source elliptic curve cryptography" on one occasion I found many of my pages with references to academic signature, but not the homepage of academic signature itself. This is a clear indication that is has been selectively suppressed in the results. The positive rank in fastbot or yandex is probably superimposed by a more powerful censorship caused by an unknown entity, which is also included as primary source. It seems that in my test in this meta search engine, specific suppression by an unnamed source is taken into account at least as strongly as positive indexing by others, which are named. 30 Minutes later, the same test revealed another result and drew information from entirely different primary search engines...
To be honest - to me this stinks, and I recommend to avoid metager altogether. This is in stark contrast with metager's public image.
In reponse to the search "open source elliptic curve cryptography" on may 22nd 2017 metager listed this page #1, claiming to have come to this link via yahoo.

 

To sum it all up:
Regarding the non commercial homepage of Academic Signature, I see friendly behavior and no sign of manipulation in:
(non Us)Google, Russian Yandex and Chinese Baidu, now also on Bing, Duckduckgo and Metager.

In US-TOR-exit node google.com results, I see inconsistent returns. Mostly, my requests for "open source elliptic curve cryptography" via US-TOR-exit nodes are blocked(repetitive capcha loops), sometimes they are processed. When processed, I had results with links to academic signature absent, sometimes they were included at reasonably high rank(mostly on page 1).


Fastbot.de and Disconnect.me appear to be almost broken in general, if judging by the overall spectrum also of the other search results.