So why are we giving away $1,000? Well, we need help making Twitturly even better. We want to be able to have the results on Twitturly automatically be grouped together by the semantics of the results. To claim the $1,000 prize you need to be a programmer with the skills to pull it off the majority of the time. It doesn’t need to be perfect, but it should be close.

Here is the problem:

Currently, the number one and two result are, “goosh.org - the unofficial google shell”.

Now, you and I can tell that both of these links are going to take you to a site that shows either exactly the same thing or close to the same thing. So what I want to do is make it so that the second result appears underneath the first result as a sub-item.

To help with this programming challenge, I have decided to export the final array (slightly modified to remove proprietary ranking information) and serialize this for you guys to play with. It can be accessed here and put into a variable in your PHP scripts. To get the array how it is supposed to be, you must run PHP’s unserialize function on it. Here is the more readable and prettified version.

What I am hoping for is a function that I can send the array to. Then the function will look at the titles and descriptions to determine if it is similar to any of the results before it. It only needs to check the see if the item is related to items that appear before the current position in the array. The function would then modify that object and add the parent’s urlid to a object value of “parent” (which should be the first related result). When the function is done checking, just send back the entire modified object and we’ll do the rest.

In addition to grouping Goosh, there are a few others in the array that is included in the above sample that should be grouped. For example, #4 and #6 (#3 and #5 if you are looking at the PHP array ID’s) should be grouped. #8, #16, and #38 should all be grouped together as well (in this case, #8 should be the “parent” for both #16 and #38).

All we need is the PHP function. Our GUI guy can take care of making it look presentable (Tony, thats you!)

How to solve this issue:

Thats up to you! If you want to use one of the various API’s online, it should be free are really close since we don’t have the backing to pay for an expensive API. Ideally it will be all done in your PHP code however we understand that there maybe API’s that we can tap into to help out on this, and we are willing to give them a try if they do a good job. It should look at the semantics of the text because we want all similar items to be grouped together, even if the words are not exactly the same.

About the prize:

After all entries have been reviewed, we will pick the one that returns the best matches the most often. Once we have selected a winner (by July, 7th, 2008), they’ll get the $1,000. We understand that it’s not $100,000 like some of the challenges from companies with Venture Capital have done, but Twitturly has had no financing at all and hey, if you add the decimals at the end $1,000.00 kind of looks like $100,000! In addition to the $1,000 that we are giving away, the programmer will also get the warm fuzzy feeling that he helped make Twitturly better for everyone.

When it ends:

This contest expires on July 1st, 2008. The winner will be chosen by by July, 7th, 2008.

Please post a comment

If you are interested in participating in this contest or you have any questions, please comment below. Those that say that they are going to work on a solution will be emailed instructions on how to submit their code.

Thank you and good luck!

—-

UPDATE (06/19/2008):
Since many people are asking for the URLs to be included, I have attached two more files that can be used. One is an updated serialized results file, the other is a few PHP functions that can be used to get the data in the new results file into your PHP script.

New Results File: Download
New PHP Functions: Download

I just wanted to write a quick post to publicly ponder the question, why does it seem that Twitturly is the only site of our kind that realized that if you are monitoring the twitter public time line that you would need to include spam filtering and learning techniques?

Lately twitter has been getting a ton of press and is gaining in popularity among the average Internet users quite quickly. As such, the spammers have already determined that they could spam twitter users and get people to their site - which frankly, sucks.

Twitter is a service for getting tweets out, not for spam prevention. It would be nice of them to be able to determine if a message was spam and not let it hit the public timeline, but they have far more important things to do that add that capability to their systems.

I won’t name any names, but sadly, in the past few days, most of the services that are similar to twitturly have suffered from these spammers. Often times, retweeting the original spam message and making it even worse.

So I ask this question to anyone building anything off of twitter, if you haven’t built in your own spam filtering services for twitter that your service can use to prevent abuse of it, then why not?

Feel free to answer in the comments below.

Here is something quick that I threw together regarding the new design. It is my first one so I apologize about the quality. And you’ll need to turn up you volume because it is a lot lower than I thought it was when i created it. Sorry in advance.

So without further ado, here you go:

Today, May 9th the guys over at the Mashup Awards awarded Twitturly with a “Mashup of the Day” award.

We’d like to say thanks for giving Twitturly this award! We are truly honored.

I am not quite sure how well this will turn out, but I have decided to go ahead and setup a blog for Twitturly.

Each time that we do something interesting that we feel we should all of the Twitturly users know about, I, or in the future, someone else will post about it to let you guys know.  Don’t worry, if you are following us on Twitter, we will probably post it there too.

I also intend on occasionally posting little tidbits of information that we gather from our service, like did you know that in the last 24 hours, over 46,000 unique expanded URLs were posted to Twitter?  And for those roughly 46,000 URLs, there were over 60,000 tweets?

So, I hope that you enjoy reading it as much as I hope I enjoy doing the Official Twitturly Blog!