I was recently working on a project in which I was using JavaScript to output HTML (my Google History download tool). I needed to escape special HTML characters in the string, in order to prevent problems with the output being mangled by any < followed by > characters in the strings I was outputting. I created a simple String method that emulates PHP’s htmlentities function:

String.prototype.htmlEntities = function() {
	return this.valueOf().replace(/&/g, '&amp;').replace(/</g, '&lt;').replace(/>/g, '&gt;').replace(/"/g, '&quot;');
}
foo = "<pre>This string has HTML tags<pre>";
bar = foo.htmlEntities(); // bar contains: &lt;pre&gt;This string has HTML tags&lt;pre&gt;

It’s quite rudimentary, and simply chains together a few replace calls to search/replace &, <, >, and " characters with the appropriate HTML escape characters. Unfortunately JavaScript provides no htmlentities equivalent, but it is easy enough to recreate via a simple String method.

If you are encoding strings to be used in URLs, you’ll want to use the escape(), encodeURI(), and encodeURIComponent() JavaScript functions. Each function varies slightly in the characters it converts. If you want, you can convert them into String methods rather than use them as functions:

String.prototype.escape = function() {return escape(this.valueOf());}
String.prototype.encodeURI = function() {return encodeURI(this.valueOf());}
String.prototype.encodeURIComponent = function() {return encodeURIComponent(this.valueOf());}

special_chars = '~!@#$%^&*(){}[]=:/,;?+\'"\\';
str1 = special_chars.escape(); // str1 contains: %7E%21@%23%24%25%5E%26*%28%29%7B%7D%5B%5D%3D%3A/%2C%3B%3F+%27%22%5C
str2 = special_chars.encodeURI(); // str2 contains: ~!@#$%25%5E&*()%7B%7D%5B%5D=:/,;?+'%22%5C
str3 = special_chars.encodeURIComponent(); // str3 contains: ~!%40%23%24%25%5E%26*()%7B%7D%5B%5D%3D%3A%2F%2C%3B%3F%2B'%22%5C

Writing methods for built-in JavaScript may be a bit of overkill, but it’s a nice way to demonstrate how easy it is to create new JavaScript methods for the String object. To learn more about these URL escaping functions, be sure to visit: http://xkr.us/articles/javascript/encode-compare/.

Perhaps a better example for another String manipulation method you may want, is a method to remove tags:

String.prototype.stripTags = function() {
	return this.valueOf().replace(/<[^>]+>/g, "");
}
foo = "<pre>This string has HTML tags<pre>";
bar = foo.stripTags(); // bar contains: This string has HTML tags

A couple of more tips when creating String methods in JavaScript:

• When you’re writing new methods for strings, be sure to refer to this.valueOf() to obtain the value of the string object itself
• If you create any variables within your new methods, be sure to use the var directive to declare/assign the variables. If you do not use the var directive, the internal variables will have a global scope, which can wreak havoc on your code!

The last tip really applies to any object methods and custom JavaScript functions you create. Although it can be convenient that JavaScript does not require explicit declaration of variables and types, it can be problematic to debug if you do not declare variables within object methods and global functions.

Any variables declared in an object method or function without the var directive will be global. If you reuse variable names in multiple methods/functions (such as “x” or “i” for loop variables, for example), you may experience very unexpected results if you do not explicitly declare them with the var directive.

The trick is to use the RANK() and OVER clauses. When combined together, RANK() and OVER can perform aggregation in a query without having to use the GROUP BY clause. This allows you to obtain aggregation information for individual rows in a result. Unfortunately, this does not work in MySQL, but it does work in both Oracle and T-SQL (transact SQL, which is what MS SQL Server uses).

For example, let’s say you have the following table called employees:

You want to select only those individuals that have the highest salary, by department. In MS Access, you could do this with the First() function:

SELECT
	Department,
	First(Employee_Name) as Employee_Name,
	First(Salary) as Salary
FROM
	(SELECT
		*
	FROM
		employees
	ORDER BY
		Department,
		Salary DESC
	) AS [Ordered Salaries]
GROUP BY
	Department

This will create an inner query that sorts by department and then descending salary. The outer query groups by department and selects the first salary and name. Because the inner query is sorted, the First() function returns the employee with the highest salary for each department.

You can simulate the First() function in SQL Server and Oracle, with RANK() OVER. Here is what the inner query would look like in SQL Server (T-SQL):

SELECT
	employees.Department,
	employees.Employee_Name,
	employees.Salary,
	RANK() OVER(PARTITION BY Department ORDER BY Salary DESC) AS Ranking
FROM
	employees

This query would return:

Notice how the RANK() OVER(PARTITION BY) clause provides aggregated information for rows in the result, without having to use the GROUP BY clause. This can be a very powerful tool, especially considering that other aggregation functions such as MIN(), MAX(), SUM(), etc. can be used with OVER(PARTITION BY).

You can probably already see where this is going. The PARTITION BY Department portion of the query aggregates the table by department, and the results are then ordered descending by salary.

To get the final result we desire, all we need to do is select the rows of the inner query where the Ranking is 1. Here’s what the final query looks like:

SELECT
	Department, Name, Salary
FROM
	(SELECT
		employees.Department,
		employees.Name,
		employees.Salary,
		RANK() OVER(PARTITION BY Department ORDER BY Salary Desc) AS Ranking
	FROM
		employees
	) AS [Ranked Salaries]
WHERE
	Ranking = 1
GROUP BY
	Department

And the final result:

Although the First() function may be a bit more intuitive in Access, emulating it with RANK() OVER(PARTITION BY) works just as well. The syntax for Oracle PL/SQL is exactly the same:

RANK() OVER(PARTITION BY Some_Column ORDER BY Some_Other_Column)

You just need to put the column(s) you want to group after the PARTITION BY and then sort the data accordingly. After you use it a few times, you’ll get the hang of it. You can also use this method to obtain the top results for a category. You would simply just select records with a ranking greater than or equal to the top number of results you want to see. Perhaps that will be another tutorial for another time.

Stop words are typically things like conjunctions, prepositions, etc. Typically, a developer would want to remove stop words from a string, in order to extract keywords from it. I found a stop word list after a quick search, and used it in my method. The method I wrote actually extends the JavaScript String data type, so it can be applied to any string. You just have to be sure to define the prototype method before calling it.

Once you’ve defined it, you just call the method like this:

foo = "here is a string that has some stop words in it";
bar = foo.removeStopWords();
// The value of bar will be "string stop words"

The method works by first defining a very large array with stop words in it. The string itself is split by whitespaces within the string, creating an array of contiguous non-whitespace characters. There are then two loops for processing the words.

The outer loop cycles through all of the words in the string, and the inner loop cycles through all of the stop words. The inner loop performs a comparison between the current word and current stopword. If the word matches the stop word, all instances of it are removed from the main string.

Here’s the code:

/*
 * String method to remove stop words
 * Written by GeekLad http://geeklad.com
 * Stop words obtained from http://www.lextek.com/manuals/onix/stopwords1.html
 *   Usage: string_variable.removeStopWords();
 *   Output: The original String with stop words removed
 */
String.prototype.removeStopWords = function() {
	var x;
	var y;
	var word;
	var stop_word;
	var regex_str;
	var regex;
	var cleansed_string = this.valueOf();
	var stop_words = new Array(
		'a',
		'about',
		'above',
		'across',
		'after',
		'again',
		'against',
		'all',
		'almost',
		'alone',
		'along',
		'already',
		'also',
		'although',
		'always',
		'among',
		'an',
		'and',
		'another',
		'any',
		'anybody',
		'anyone',
		'anything',
		'anywhere',
		'are',
		'area',
		'areas',
		'around',
		'as',
		'ask',
		'asked',
		'asking',
		'asks',
		'at',
		'away',
		'b',
		'back',
		'backed',
		'backing',
		'backs',
		'be',
		'became',
		'because',
		'become',
		'becomes',
		'been',
		'before',
		'began',
		'behind',
		'being',
		'beings',
		'best',
		'better',
		'between',
		'big',
		'both',
		'but',
		'by',
		'c',
		'came',
		'can',
		'cannot',
		'case',
		'cases',
		'certain',
		'certainly',
		'clear',
		'clearly',
		'come',
		'could',
		'd',
		'did',
		'differ',
		'different',
		'differently',
		'do',
		'does',
		'done',
		'down',
		'down',
		'downed',
		'downing',
		'downs',
		'during',
		'e',
		'each',
		'early',
		'either',
		'end',
		'ended',
		'ending',
		'ends',
		'enough',
		'even',
		'evenly',
		'ever',
		'every',
		'everybody',
		'everyone',
		'everything',
		'everywhere',
		'f',
		'face',
		'faces',
		'fact',
		'facts',
		'far',
		'felt',
		'few',
		'find',
		'finds',
		'first',
		'for',
		'four',
		'from',
		'full',
		'fully',
		'further',
		'furthered',
		'furthering',
		'furthers',
		'g',
		'gave',
		'general',
		'generally',
		'get',
		'gets',
		'give',
		'given',
		'gives',
		'go',
		'going',
		'good',
		'goods',
		'got',
		'great',
		'greater',
		'greatest',
		'group',
		'grouped',
		'grouping',
		'groups',
		'h',
		'had',
		'has',
		'have',
		'having',
		'he',
		'her',
		'here',
		'herself',
		'high',
		'high',
		'high',
		'higher',
		'highest',
		'him',
		'himself',
		'his',
		'how',
		'however',
		'i',
		'if',
		'important',
		'in',
		'interest',
		'interested',
		'interesting',
		'interests',
		'into',
		'is',
		'it',
		'its',
		'itself',
		'j',
		'just',
		'k',
		'keep',
		'keeps',
		'kind',
		'knew',
		'know',
		'known',
		'knows',
		'l',
		'large',
		'largely',
		'last',
		'later',
		'latest',
		'least',
		'less',
		'let',
		'lets',
		'like',
		'likely',
		'long',
		'longer',
		'longest',
		'm',
		'made',
		'make',
		'making',
		'man',
		'many',
		'may',
		'me',
		'member',
		'members',
		'men',
		'might',
		'more',
		'most',
		'mostly',
		'mr',
		'mrs',
		'much',
		'must',
		'my',
		'myself',
		'n',
		'necessary',
		'need',
		'needed',
		'needing',
		'needs',
		'never',
		'new',
		'new',
		'newer',
		'newest',
		'next',
		'no',
		'nobody',
		'non',
		'noone',
		'not',
		'nothing',
		'now',
		'nowhere',
		'number',
		'numbers',
		'o',
		'of',
		'off',
		'often',
		'old',
		'older',
		'oldest',
		'on',
		'once',
		'one',
		'only',
		'open',
		'opened',
		'opening',
		'opens',
		'or',
		'order',
		'ordered',
		'ordering',
		'orders',
		'other',
		'others',
		'our',
		'out',
		'over',
		'p',
		'part',
		'parted',
		'parting',
		'parts',
		'per',
		'perhaps',
		'place',
		'places',
		'point',
		'pointed',
		'pointing',
		'points',
		'possible',
		'present',
		'presented',
		'presenting',
		'presents',
		'problem',
		'problems',
		'put',
		'puts',
		'q',
		'quite',
		'r',
		'rather',
		'really',
		'right',
		'right',
		'room',
		'rooms',
		's',
		'said',
		'same',
		'saw',
		'say',
		'says',
		'second',
		'seconds',
		'see',
		'seem',
		'seemed',
		'seeming',
		'seems',
		'sees',
		'several',
		'shall',
		'she',
		'should',
		'show',
		'showed',
		'showing',
		'shows',
		'side',
		'sides',
		'since',
		'small',
		'smaller',
		'smallest',
		'so',
		'some',
		'somebody',
		'someone',
		'something',
		'somewhere',
		'state',
		'states',
		'still',
		'still',
		'such',
		'sure',
		't',
		'take',
		'taken',
		'than',
		'that',
		'the',
		'their',
		'them',
		'then',
		'there',
		'therefore',
		'these',
		'they',
		'thing',
		'things',
		'think',
		'thinks',
		'this',
		'those',
		'though',
		'thought',
		'thoughts',
		'three',
		'through',
		'thus',
		'to',
		'today',
		'together',
		'too',
		'took',
		'toward',
		'turn',
		'turned',
		'turning',
		'turns',
		'two',
		'u',
		'under',
		'until',
		'up',
		'upon',
		'us',
		'use',
		'used',
		'uses',
		'v',
		'very',
		'w',
		'want',
		'wanted',
		'wanting',
		'wants',
		'was',
		'way',
		'ways',
		'we',
		'well',
		'wells',
		'went',
		'were',
		'what',
		'when',
		'where',
		'whether',
		'which',
		'while',
		'who',
		'whole',
		'whose',
		'why',
		'will',
		'with',
		'within',
		'without',
		'work',
		'worked',
		'working',
		'works',
		'would',
		'x',
		'y',
		'year',
		'years',
		'yet',
		'you',
		'young',
		'younger',
		'youngest',
		'your',
		'yours',
		'z'
	)

	// Split out all the individual words in the phrase
	words = cleansed_string.match(/[^\s]+|\s+[^\s+]$/g)

	// Review all the words
	for(x=0; x < words.length; x++) {
		// For each word, check all the stop words
		for(y=0; y < stop_words.length; y++) {
			// Get the current word
			word = words[x].replace(/\s+|[^a-z]+/ig, "");	// Trim the word and remove non-alpha

			// Get the stop word
			stop_word = stop_words[y];

			// If the word matches the stop word, remove it from the keywords
			if(word.toLowerCase() == stop_word) {
				// Build the regex
				regex_str = "^\\s*"+stop_word+"\\s*$";		// Only word
				regex_str += "|^\\s*"+stop_word+"\\s+";		// First word
				regex_str += "|\\s+"+stop_word+"\\s*$";		// Last word
				regex_str += "|\\s+"+stop_word+"\\s+";		// Word somewhere in the middle
				regex = new RegExp(regex_str, "ig");

				// Remove the word from the keywords
				cleansed_string = cleansed_string.replace(regex, " ");
			}
		}
	}
	return cleansed_string.replace(/^\s+|\s+$/g, "");
}

Recently, I started playing in a fantasy social media stock exchange, called Empire Avenue. It took me a while to get a handle on what moves the prices, and I’m still learning. With some help from Chris Pirillo, I think I’ve finally got a good handle on it and he’s helped me start a tool that can help you build a sound Empire Avenue portfolio too.

I first came to learn about Empire Avenue when Jesse Stay made a few blog posts about it where he writes about Empire Avenue as a catalyst for the new economy and the new frontier of capitalism. Both articles were excellent, but my interest hadn’t quite piqued until my good friend Bwana posted his thoughts on Empire Avenue and he referred me to Chris Pirillo’s tips on Empire Avenue.

Two Ways to Make Eaves

Eaves are the Empire Avenue (virtual) currency. Being a numbers guy and with data analysis being my day job, I wanted to learn how to play the Empire Avenue exchange. I figured I could probably put together some tools to earn some virtual money, just as I did in Urban Rivals.

When it comes down to it, there are two ways to make Eaves in Empire Avenue with your investments:

• By earning dividends from what I will henceforth refer to as stocks (which are actually people or companies – i.e. entities with social networking accounts)
• By buying stocks for a low price and selling at a higher price

Fundamentally speaking it’s pretty simple, but in order to determine what constitutes a good investment, you need to learn what drives both dividends and stock prices. It became very clear to me when Chris Pirillo told me:

• Stock price = buying interest from others
• Dividends = your (social networking) activity (both on and off-site)

Dividend Yield

Chris Pirillo’s strategy is to identify stocks that provide a high dividend, relative to the stock price. I like this strategy very much, because it is quite conservative. If you find a good dividend producer, the stock price could remain stagnant or even drop, and you still make money. While you own the stock, you will continue to earn daily dividends, which come from the daily social networking activities of the person/company.

This strategy also applies to real stock exchanges, where investors might purchase a stock that has an attractive dividend yield. Dividend yield is calculated by dividing the dividends paid per share by the stock price. It’s a good measure to use to decide whether or not a stock might be a good investment, in terms of the dividends it will provide relative to the purchase price of the stock.

Because I was attracted to Chris’ strategy of focusing on high dividend yielding stocks, I started developing my own Chrome Extension to provide some investment analysis on Empire Avenue stocks. Chris provided a great deal of help in developing the tool, with suggestions on what to add. In addition to providing dividend yields, it also provides break even information (which I will explain in a bit) as well as current profitability info for the stocks you currently own.

Capital Appreciation

Just as one might buy real life stocks low and sell high, that same concept applies to Empire Avenue. In real life stock markets, this is referred to as capital appreciation. You can certainly use dividend yield to decide whether or not a stock is still relatively inexpensive compared to the dividends it provides. Chris also uses this as an indicator for whether or not a stock has room to grow in price.

I think this is a decent strategy as well, however, I would not rely on it alone. The price of a stock is driven by the demand to buy it. If a lot of people want to buy it, the stock price will move up. By the same token, if the stock has appeared to plateau, it may drive individuals to sell it which in turn would drive the stock price down. The speculative nature of these ups and downs can make investing based on price alone somewhat risky.

Until more tools such as mine and Eddie Ringle’s (I will likely incorporate my calculations into his extension eventually, as he has his open sourced and posted to Git) come out and provide additional valuation metrics and analysis, there is also the human nature of thinking because the share price is high the stock is expensive, which is mistake many would-be investors make. One cannot value a stock as inexpensive or expensive based on share price alone. It must be placed into context with valuation metrics such as dividend yield.

For example, my stock e(GEEKLAD) is currently trading for 13.620 and Chris’ stock e(PIRILLO) is trading for 156.32. If you were to value our stocks on share price alone, you would say mine is cheap and Chris’ is expensive. However, if you used dividend yield to valuate our stocks, Chris’ could be considered much cheaper. His provides more than a 1% dividend yield, whereas mine does not even provide 0.50% dividend yield.

On the other hand, there is still that human factor where most individuals would have the tendency to say Chris’ stock is more expensive than mine, and I would suspect that at his current price his stock probably won’t appreciate as fast as mine. Also, because he has many more outstanding shares, it would take much more volume for his stock to move rapidly than it would take for my stock to move. As the outstanding shares increase, the relative volume will probably decrease, so you’ll see slower acceleration in the stock price.

My Chrome Extension

My plugin provides you with the following information when you either view a stock’s profile page (when you are logged into Empire Avenue):

• Daily Dividend Yield: This the calculation of the daily dividend per share divided by the share price. In Chris’ experience, 1% is a good yield, 2% is extremely good, and 3% is nearly unheard of.
• Approximate Dividend Pay Back (Days)This is how long it would take you to earn back the money you paid to buy the stock, and cover (an assumed) 5% commission as well as the 5% commission you would need to pay when you sell it.
• Current Break Even: This is the share price the stock would have to increase to, in order to cover the 5% purchase commission and 5% sale commission if the stock were purchased at the current price.

In addition, for stocks you already own you will also see:

• Your Average Purchase Price: This is your average purchase price, excluding commissions.
• Your Breakeven Price: This is where the stock price has to be for you to recoup your purchase price, purchase commission (assumed at 5%) and sale commission as well.
• Your Real Gain per Share: This is how much you’ve actually made per share, taking account your purchase commission and your sale commission if you were to sell at the current share price.
• Your Real Total Gain: This is how much you’ve actually made in total, taking into account commissions for the purchase and sale and your average purchase price, were you to sell the price at the current share price.

Here’s a screenshot of my investment in e(BWANA):

To see this in action, visit your portfolio/lists at http://www.empireavenue.com/influencer/portfolio, then click on any stock to view the stats as you see them in the screenshot above. You can also see those stats if you visit a stock profile page. For profile pages of stocks you own, you’ll see your profitability stats.

It is very important to make sure you cover your purchase/sale commissions when you trade, otherwise you could very well sell a stock you think you’ve made money on, but you won’t be able to cover the 10.5% or so you’ll pay in commissions. Say what? Why 10.5%??? To calculate the gain, you can take 105% and divide it by 95%, which gives you a little over 110.5%. That 10.5% represents the gain from your purchase price that you need to cover your purchase and sale commissions.

If you make a hot buy that has a higher purchase commission, your break even is even higher. Unfortunately, neither the website or API (from what I can tell) provide any info on commissions you’ve paid in your transactions, so I have to assume you paid 5%. If you paid 30% commission and pay 5% on the sale, your pay back would be 36.8% above your purchase price (130% divided by 95%).

So do not be alarmed when you view a stock in your portfolio and my plugin says you currently have a loss. It’s because the stock hasn’t moved high enough to cover your commissions. However, if you have a stock that yields 1% dividends, your dividends will have covered your commissions in 11 days or so. In a future version of the extension, I want to account for accumulated dividends to get a true ROI calculation, which would be:

Sale Price – Sale Commissions – Purchase Price – Purchase Commissions + Accumulated Dividends

What Chris previously referred to as ROI (previous to the discussions I had with him), is actually dividend yield. ROI would be calculated as I have stated in my formula above, for investments currently being held. If I can get to a true ROI calculation that includes accumulated dividends, my extension will be that much better.

So, give it a shot and download the Empire Avenue Investment Analyzer Chrome extension download the P³ for Empire Avenue Chrome extension. Let me know what suggestions you may have by leaving comments here, or if you run into any bugs. Feel free to ask me any questions about investing on Empire Avenue as well, and I’ll be glad to try to answer them. Good luck and happy investing!

If you like my extension, feel free to show your appreciation and pick up a few shares of e(GEEKLAD).

Some time ago, a friend got me into an online card game called Urban Rivals. It is similar to other card games such as Magic the Gathering, but played online either via iPhone/iPad app or in your browser. One aspect of the game that quickly got my attention (other than the game itself), was the card marketplace where you can buy and sell playing cards.

It didn’t take long for me to realize that it was an inefficient market. Profitable trades presented themselves quite regularly, and with a bit of automation it would be easy to take advantage of the market. I proceeded with developing a Google Chrome extension that would allow me to do so, and it worked quite well.

Over the course of a few days, I had it up and running, automating both buying and selling of cards. Essentially, it continually scans the marketplace until a profitable buy appears, buys the card (or cards), and immediately posts them for sale at one clintz (the in-game currency) below the lowest price. I told a friend of mine about it, and with some trepidation at first, he decided to try it out and helped me test and tweak it. Eventually, he realized it worked great and used it extensively as I did.

When it was all said and done, I had accumulated probably over 6 million worth of clintz and cards, having started with just a few thousand. My friend had probably accumulated on the order of about 5 million. Ultimately, my friend and I were both banned. I told him if ever I did get banned, I would likely release the extension into the wild to see what sort of havoc it might wreak on the marketplace. Well, I did get banned, so here we are. Even before I was banned, I had a morbid curiosity to learn what might happen if I were to give it to more than just one or two other players. Having been banned was just the excuse I was looking for, as I (and my friends) no longer have a vested interest in keeping it a secret.

Words of Wisdom in Using the Urban Rivals Market Bot

First and foremost, Use at Your Own Risk.
If the good folks of Urban Rivals find out you’re using a bot, they will ban you. If you use it in moderation, you may not get caught. Of course, I can make no guarantees. If you get banned for using this, don’t blame me as you’ve been warned!

Be very careful if you set up a separate account.
My friend set up an account for using the bot, in case if he got banned, so he would still have the original account. Unfortunately this plan backfired on him, as he was banned not for the bot but both accounts were banned for having duplicate accounts.

If you do use decide to set up an additional account, do not ever connect to them from the same internet connection. You could try using Tor to connect from an anonymous internet connection for the bot account, and use your regular internet connection for your regular account. I haven’t tried this, but this method of using multiple accounts would probably work without a problem. You just have to be very careful you never connect via the same method/connection.

It may not work for long, or at all at this point.
Since I no longer have a valid Urban Rivals account, I have no mechanism to see if it even still works. The last time I used it, it worked without issue so it’s likely it still works, but I make no guarantees. So that my “legacy” may still live on, I’ll include the source code as well.

If you’re good with JavaScript, you can probably figure out my mess of code and fix things if they’ve broken. If you do fix any bugs you see, feel free to send me the new code and I’ll post it here.

Be prepared for personal attacks.
Because the bot works by looking for good deals, occasionally it picks up deals that were posted in error. Sometimes people forget a 0, and end up posting a card worth thousands for hundreds, or even hundreds of thousands for tens of thousands. The bot will pick these up for a song and repost them for sale and make 10x profit. Of course, when this happens it sometimes solicits inflammatory language from those that posted the price in error.

I would gladly exchange the card back to players that asked politely and patiently. Either I would tell them to sell me a cheap card for slightly less than the value of the card so I could still make a profit, or sell the card back to them (if I still had it) for a profit. I would recommend you do the same, if you do not wish to draw the attention of the UR gods. If you want to remain even more transparent, you may even wish to throw the impolite requestors a bone as well.

I would also recommend disabling your online presence for your UR login. I think in many cases, the players I pissed off thought I was online, when in fact it was simply that my browser window was open conducting market transactions on my behalf. In some cases, I was receiving many messages from incensed players within minutes of each other, but I did not see the messages for many hours later and sometimes even days.

When I finally realized my online presence was the reason I was receiving so many messages so quickly from the same angry players, I disabled my online presence. This helped quite a bit to quell the angry messages coming in rapid succession, but they did still happen. Be nice to those who make mistakes in their market transactions and ask you to sell their card back to them.

Instructions

1. If you don’t have it, download Google Chrome, the best web browser known to man

2. Download and install the Urban Rivals Market Bot Extension for Google Chrome
3. Configure the bot however you like:
   • Enable Profitability Filter: This enables/disables the filtering of the market to only show profitable trades, according to the profitability settings.

   • Enable Stars Filter: You can filter the market to only show cards with a certain number of stars
   • Min and Max Price: The minimum and max price to filter on. Cards less than the minimum or more than the maximum will be removed from the market list.
   • Min Profit: The minimum amount of profit to make on the purchase and sale of a card, if sold for 1 clintz less than the lowest listed card. Any cards that do not meet this criteria will not be displayed in the market.
   • Min Percent Profit: The minimum percent profit to make on the purchase/sale. Cards must meet both the min profit and min percent profit criteria to be displayed.
   • Autobuy: Buy cards that meet the profit filter criteria. Without the profitability filter enabled, this does nothing.
   • Autosell: Sell cards that are autobought. Without autobuy enabled, this won’t do anything. If disabled while autobuy runs, cards will be accumulated in a profitable manner, but not sold.
   • Refresh Delay: How many seconds to wait before refreshing the market page to try autobuying again. You can set it to 0. If you set it too fast, you’ll probably get a captcha, at which point the bot will stop working until you manually clear the captcha.
   • Max # of Cards to Buy: Sometimes there are profitable trades where there is a big breakpoint in the price of cards. You can buy perhaps the two or three cheapest cards, and sell at the price of the remaining cheapest card. If you want to be conservative, just leave it at 1.
   • Max Premium on Sale: Sometimes players buy up a lot of cards to raise the lowest price so they can sell the cards they buy at a higher price. This is a safeguard to avoid buying/selling a card when this has happened, otherwise you could fall victim to this market manipulation tactic. This is how far you are willing to resell a card marked up above the lowest price of the day, week, and month. If you set it too high, you could end up paying too much for a card and you won’t be able to sell it profitably. I typically ran it at 10 percent, which served me well.
4. Visit the first page of the UR market. Be sure to sort the market by date, with the most recent listings first (this is the default, so you really shouldn’t have to change anything). The page will continually refresh according to the refresh delay setting, and as profitable listings are found it will attempt to buy them and resell them 1 clintz lower than the cheapest. Please note, that it will only deal with cards with maxed experience.

The Autosale of Doubles Feature
One last note on using the plugin. If your sales expire, you may end up with many doubles in your collection. I eventually grew tired of manually selling all of my doubles, so I also added functionality to autosell doubles when visiting my collection page. When you visit your collection page, if you have doubles you’ll be prompted if you want to sell them.

If you use this feature to sell a lot of doubles at once (perhaps 50 or more), it could result in a temporary ban from using the market. This is a bit more severe than being captchad, as you can’t do anything but wait on the order of about 7-8 hours to access the marketplace again. Don’t panic if it does happen, but you won’t be able to access ANY market pages (even your pending sales, sold cards, etc.) until enough time has passed. I ultimately wanted to make this a feature that could be enabled/disabled in the settings popup, but I never got around to implementing that.

A Small Request

If someone could do me a favor, please contact mrdisco and tell him about my bot. He went on a rant in the forums to tell everyone what an asshole I am (scroll to the bottom of this post to read it). Someone please send him the link to this post. At least he can recoup his clintz using my bot, and even make a good bit more.

I attempted to contact him via UR messaging to tell him I would have gladly refunded him, if he had given me a chance to respond. He sent me a barrage of messages in a span of maybe an hour, probably while I was sleeping or at work, so I never got to respond. By the time I was able to respond, he added me to is ignore list so I couldn’t message him on UR although I wanted to.

At the time, I decided it best I not even try to explain myself in the forum. He obviously had no interest in hearing what I had to say since he never even gave me the chance to explain. At this point, I think this post is sufficient explanation and perhaps he may even no longer feel ill will.

Final Thoughts

Unfortunately, I was banned from UR (I’m sure mrdisco will be happy about that) so I am not able to post this in the UR forums, so here it is on my blog. If any discussion does take place in the UR forums about it, please feel free to repost it here. I’m very curious to know what people might say about it, if anything at all.

Oh, I almost forgot the source code!

Download the Urban Rivals Market Bot Source Code

The code is ugly but it works (at least it did, when I last used it). It’s all screen scraping with regular expressions in JavaScript. If you know JavaScript, you can probably figure it out for the most part. If you know regular expressions as well, then you can easily bend it to your will. If you do, feel free to reach out to me if you keep the code working and up-to-date.

Enjoy! Let me know if you have any questions about the plugin. Unfortunately, without a UR account I really can’t do anything about fixing bugs if there are any, but perhaps there’s a JavaScript/regular expression guru out there that can take hold of the Urban Rivals Market Bot torch.

YQL is a truly amazing web API.  With its server-side JavaScript and the ability to quickly scour the web for information, there is very little it cannot do.  I decided it was the perfect platform for putting together a simple API for goo.gl.

Because the Google Toolbar was already written in JavaScript, implementing it in YQL was quite easy with the server-side JavaScript. It wasn’t long, before I put together my own open data table for goo.gl.

I submitted my open table to the git repository for open data tables, and shortly after my goo.gl open data table was included. It’s really pretty simple. The YQL you would use to get the short URL for my blog would be:

SELECT * FROM google.goo.gl WHERE url="http://geeklad.com"

The great thing about YQL is that you can obtain the data as XML or JSON, and specify a JSON callback function. So that means it’s very easy to use in JavaScript and develop cross-domain JavaScript. I put together a simple bookmarklet that uses my public YQL table and provides a goo.gl URL for the current page. To try it out, just drag and drop the link below to your browser’s shortcut bar:

goo.gl page

Here’s the JavaScript code from the bookmarklet:

var YQL_url='https://query.yahooapis.com/v1/public/yql?format=json&env=store://datatables.org/alltableswithkeys&q=';
var YQL_statement='SELECT * FROM google.goo.gl WHERE url=\''+document.location.href+'\'';
var YQL_script_url=YQL_url+encodeURIComponent(YQL_statement)+'&callback=yql_cb';
function yql_cb(response){
	short_url=response.query.results.result;
	prompt('Here is your short URL (be sure to copy to your clipboard): ', short_url);
}
var yql_script=document.createElement('script');
yql_script.src=YQL_script_url;
document.getElementsByTagName('head')[0].appendChild(yql_script);

Pretty simple. Basically it works by putting together the YQL statement w/ the URL of the current page, builds the proper YQL API URL for it, and finally inserts a JavaScript script with the call to the callback function that displays the URL of the current page in a prompt. You’ll have to manually copy it to your clipboard. Enjoy!

Google Fusion Tables is a web application that allows you to import, manipulate, and generate reports from large data sets. It is designed to handle much more data than you can put into Google Spreadsheets or versions of Excel earlier than Excel 2007. Some time ago, I started fooling around with it and I managed to import a relatively large data set with nearly 150,000 records without any issues whatsoever. The interface also provides some nice ways to visualize the data.

In addition to providing some nifty visualizations, there are also many data tables that have been published for public use. For example, the 2010 Quality of Life Index:

So why do I think this could be a game changer? Yesterday, the Google Code Blog announced the Fusion Tables API. With the API it is no longer necessary to use the Fusion Tables web interface to upload, manipulate, and visualize data. Now it can be done programatically, and Fusion Tables can be used to store large data sets in the cloud. The data can be extracted using SQL, which is the gold standard for handling the data stored within databases.

Not long ago, Jesse Stay wrote about the building block web. The Fusion Tables API is a perfect example of a new set of blocks that will integrate very well (and easily) with the building block web. Persistent data storage is an essential component for the building block web, and that is just what Fusion Tables provides.

Of course, it doesn’t come without its limitations. According to the Google Fusion Tables API developer’s guide, the API can send a maximum of 5 requests per second to the Fusion Tables server. Hopefully Google will lift that constraint or increase it at some point. 5 requests per second won’t be enough for the enterprise applications that could potentially be built on top of Fusion Tables.

If you’re a PHP programmer, here’s some PHP code I’ve put together to help get you started:

I had recently been fooling around quite a bit with YQL, which does provide some built-in features to make cross-domain http requests. It can be done with the html open data table. You can try it out from the YQL Console.  Just go to the data tables section on the right side and select Data > html. Unfortunately, the built-in html table is somewhat limited.  You can only make requests using the GET method, and it does not provide any mechanism for sending/receiving HTTP headers.

Someone in the YQL forums posted a YQL feature request to add POST and other HTTP request methods to the html table. Someone responded with the suggestion of building an open data table to do just that. I then started reading the open data table documentation, and came to realize that what you can do with YQL and open data tables is nothing short of astounding. You can use server-side JavaScript to request data from multiple sources and manipulate it in any way you see fit.

I started developing my own custom open data table for the particular project I was working on at the time, and eventually the project morphed into xdRequest. Because I needed to track cookies and redirects in my little project, I realized that it would be great to develop a generic library that could handle such requests. Eventually, I more or less abandoned the previous project to develop xdRequest. Here are some of the features, as highlighted on the project page:

• Make cross-domain requests using the GET and POST methods
• Add request headers along with the requests
• Automatically collect cookies sent by the remote web server
• Automatically follow redirects and collect cookies along the way
• Properly store/transmit cookies, much like the popular server-side library, cURL
• Automatically submits hidden fields detected within forms on webpages

I also put together a relatively simple xdRequest demo. It brings in data from espn.go.com. When the page loads, it makes an xdRequest to ESPN for the University of Florida Football schedule. I’m a die-hard Gator and UF alumnus, so that’s why I decided to start the demo with the UF football schedule. When you can click the hyperlinks in the demo, xdRequests are made to ESPN to load schedules for other teams, as well as the box scores for games. It’s a simple example, but the sky is the limit as to what you can do with xdRequest.

If you’re interested in learning more about the project, be sure to check out the xdRequest project page. If you’re a JavaScript geek and would like to join, just drop me a line and I’ll gladly add you to the project. I’m kind of a n00b when it comes to object-oriented programming with JavaScript, so I could definitely use some help to clean up the code, make enhancements, find bugs, and fix bugs. This is the first “official” Open Source project I’ve ever worked on, so I’m a n00b when it comes to working on OSS as well.

If you like xdRequest and build any cool web applications with it, let me know and I’ll link to it from the project page. If you have any feature suggestions or encounter any bugs, please report them in the xdRequest issue tracker.

<script type="text/javascript" src="http://geeklad.com/tools/gadgetusers.js?url=www.somewebsite.com/somepath/yourgadgetfile.xml&cda6c1"></script>

Just be sure to replace the www.somewebsite.com and /somepath/yourgadgetfile.xml accordingly. My gadget user counter works by performing a document.write to output the number of users that are currently using your gadget, according to the iGoogle gadget directory. I have a feeling that these figures are understated or not updated very often, because according to Google Analytics I have many more users than the iGoogle gadget directory states. At least the directory figures are conservative and they do get updated with some frequency. Now you can easily inform people as to how many users are using your gadgets too! Check out my web definitions gadget page to see it in action.

]]> http://geeklad.com/display-your-igoogle-gadget-user-count/feed 2