Welcome!

Join our community of MMO enthusiasts and game developers! By registering, you'll gain access to discussions on the latest developments in MMO server files and collaborate with like-minded individuals. Join us today and unlock the potential of MMO server development!

Join Today!

Gtop100 Scraper

Custom Title Activated
Member
Joined
Nov 14, 2008
Messages
1,025
Reaction score
640
A method that returns a Map containing data scraped from gtop100, requires Jsoup.

You can download the latest version of Jsoup right
If you don't know what to do when you downloaded Jsoup, then this release probably isn't for you as it requires you to know some java to use the scraper for other things properly.

The method that does all the magic
PHP:
public static Map<Integer, GtopPosition> scrap() {
	Map<Integer, GtopPosition> gtop100 = new HashMap<>();
	long before = System.currentTimeMillis();
	try {
	    Document doc = Jsoup.connect("http://www.gtop100.com/maplestory-1").timeout(1000 * 10).get();
	    int it = 0, it2 = 0;
	    for (Element elem : doc.getElementsByClass("cell")) {
		GtopPosition position = new GtopPosition(-1, "", -1, -1);
		it++;
		for (Element elem1 : elem.getElementsByClass("link")) {
		    if (it % 2 == 0) {
			it2++;
			String name = elem1.text();
			position.name = name;
			position.pos = it2;
			gtop100.put(position.pos, position);
		    }
		}
	    }
	    it = 0;
	    it2 = 0;
	    for (Element elem : doc.getElementsByClass("cell-rT")) {
		it++;
		it2++;
		int ut = 0;
		int in = -1;
		int out = -1;
		for (Element elem1 : elem.select("label")) {
		    if (ut == 2) in = Integer.parseInt(elem1.text());
		    else if (ut == 3) out = Integer.parseInt(elem1.text());
		    ut++;
		}
		if (in == -1 || out == -1) {
		    it2--;
		    continue;
		}
		GtopPosition entry = gtop100.get(it2);
		entry.in = in;
		entry.out = out;
	    }
	} catch (Exception e) {
	    System.out.println("Scraping failed!");
	    return null;
	}
	long then = System.currentTimeMillis();
	System.out.println("Scraping done in " + (then - before) + " milliseconds!");
	return gtop100;
    }

And the gtop position class
PHP:
public class GtopPosition {

    String name = "";
    public int pos, in, out;
    
    public GtopPosition(int pos, String name, int in, int out) {
	this.pos = pos;
	this.name = name;
	this.in = in;
	this.out = out;
    }
    
    public int getPos() {
	return pos;
    }
    
    public int getIn() {
	return in;
    }
    
    public int getOut() {
	return out;
    }
    
}

i'm sure this could of been done in a better way but I suck with JQuery selectors :p
This could probably be used for other games aswell if you wish.

There is a bit of stuff you could do with this such as giving double EXP for say 10 minutes after going up a rank on gtop100 or enforcing people to vote when getting close to the next rank (80% of the way done for example)
 
Junior Spellweaver
Joined
Apr 15, 2010
Messages
147
Reaction score
8
Pretty Great release.

Still a bit new to java, so cant use, but do wish i could. Very cool incentive, and can expect this to show up around gtop soon.
 
Back
Top