|
Page 1 of 4 Searching
the Web is easy. The difficult part is to find what you are looking
for. While the search engines and mainly Google have done miracles in
making millions of pages accessible, finding what you need when you
need it is not always an easy task. And the truth is, that the ease
with each Google has made it possible to retrieve most of the
information we need, has made us lazy to search in depth, when the
stuff we are looking for is not right at the the top of the first ten
search results.
Also,
as my experience shows, the majority of users rely heavily on Google
only - if something cannot be found via Google, then it simply does
not exist, which is certainly not so. When Google cannot immediately
retrieve a piece of information, most people just conclude that only
what is retrieved exists and give up. I have often been in a similar
situation – no matter how sophisticated queries I perform, I
cannot find what I need but I am sure that it exists. Over the years
I have developed a habit to keep an archive of important URLs and
files that I stumble upon incidentally and that I might need some
time in the future, but I am clever enough to know that even if I
could make my personal mirror of the Net, this could hardly be more
useful than Google.
Also,
I am aware of the way search engines work and despite their
revolutionary achievements, I do not expect them to be perfect. I
know that even the most powerful search engines cannot index every
single page on the Web and include them in their databases. And since
search results display only information that is indexed and is in the
search engine’s database, if my stuff is not indexed and is not
included in the database, I stand no chance of finding it at all. I
have already learned that search engines might be the easiest way to
search the Web, but certainly they are not the only one. Besides,
sometimes (for very specific searches) the major search engines just
waste my and drown me in so much irrelevant information that I regret
having to use them and resort to alternative means to make my way
through the Invisible Web.
The Invisible Web
The
Invisible Web is a term I like very much because it describes
precisely the situation when I know some piece of information is on
the Web but I cannot see it. It is that vast part of the Net that
search engines do not get to (due to different reasons, as I am going
to tell next) but still can be accessed in other ways.
Maybe
it is necessary to explain that not all pages that cannot be
retrieved through the search engines belong to the Invisible Net. For
instance, the Opaque and the Dark Net are two other
places that are hidden from the world because the Opaque Net is files
that are not linked to other resources and cannot be accessed and the
Dark Web is invisible on deliberately – i.e. Corporative
networks, sites with special membership and other similar places that
do not welcome strangers. To get to Opaque and Dark Web sites, you
need to know their URL in advance (for instance from a friend of
yours) and if necessary, to have a user name and a password.
The
search ideas I am going to give you in the next sections apply to the
Invisible Web only and are unlikely to give results for the
intentionally hidden parts of the Web. But even the Invisible Web
alone is a pretty vast space. It is estimated that it is up to 500
(yes, five hundred) times the size of the Surface Web (the
part that is indexed by search engines) and the tendency is that the
Invisible Web will grow both as a percentage and in absolute figures.
And what is more, really valuable stuff is hidden in its debris.
|
