2010: This is the age of distributed computing

One mind will never cease to need the rest. Discoveries result from organized groups that communicate and collaborate as one. Today, the Internet provides for us the communication aspect of discovery. It is firmly established. However, our collaborative efforts remain scattered and unorganized. Our brains, which deal "with more than 11 million disparate pieces of information per second" (Baker 25), are disconnected. We think alone and remain so. The calculations in our brains fail to reach the black board of discovery. Sure, we can devise formulas to model happenings, but such discoveries pale in comparison to what we can find in putting the grand formulas to use. Computers, through the Internet, only spread the formulas.

Sole computers suffer from disorganization. Like us, they discover little alone. Yet, they are unique. They can be connected in a way in which our brains cannot. They can put the formulas to good use. Today, distributed computing platforms can organize our disparate dialogues and form great collaborations. Networked computers are carrying out processes undoable by people. "Around the world, computer scientists and computational biologists are designing algorithms to sift through billions of gene sequences" (Baker 160). We create the cookie cutters, the modes of discovery, and networked computers fill them with interesting revelations. Other computers are sifting through the World Wide Web itself and creating pools of useful data. Search engines like Google do just that. Most have built their own in-house distributed computing platforms. The idea is "that a webpage is important if it is pointed to by other important pages" (Langville 28). Now Google's platform continues to provide us with the discoveries of its formula.

Every large set of data has too many discoveries to count waiting inside. All computer users should be participating in distributed computing projects. Additionally, project leaders should be offering incentives to participants. Today is distributed computing's day. It will take a highly successful project to bring on such an age of collaboration. The measure of great success is widespread use. Distributed computing's golden project will be a search engine, one that rises to take on Google. People everywhere search. Such a success would affect people everywhere.

All in all, this is my plea for a distributed search engine. Help to bring search market share to someone's door. The innovations that spring from search require that more engines take on bigger challenges. There are just so many possibilities. Search engines provide relevant results because of the stellar platforms that feed their databases. Empower more platforms and drive more innovation. Even if not my own platform, help one to grow and become of consequence. It is a world of discovery out there.

~ Joshua Schwarz
~ December 31, 2009

('DiggThis’)
Bookmark and Share


Distributed Search Engine Projects
Last Updated: Jan. 5, 2010

Active Projects:
- Amagit.COM - Download page - Recent forum posts
- Faroo - Download page - Client updates history
- Grub - Project page - Recent posts
- Majestic-12 - Download page - Recent statistics
- Sciencenet - Active peers
Dormant Projects:
- Boitho - No recent statistics
- Hyperbee - Scattered web presence
- Nusearch - Scattered web presence
Software, Code, and Platforms:
- Eikon Image Search
- GPU
- Harvest
- Lucene
- Sphinx
- YaCy
General Distributed Computing Information:
- A List of Distributed Computing Projects
- Distributed Computing News