Java 4k Resources Thread

[Riven]

Server up and running... now streams the output, instead of a long delay.

[Riven]

Added kzip shrinker, using pjt33's gzip=>gz code



A.java => 13931 bytes
A.class => 11196 bytes
A.normal.jar => 6846 bytes
A.progrd.jar => 4480 bytes
A.normal.pack => 5783 bytes
A.progrd.pack => 5750 bytes
A.normal.7z.pack.gz   => 3597 bytes
A.progrd.7z.pack.gz   => 3513 bytes
A.normal.kz.pack.gz   => 3480 bytes
A.progrd.kz.pack.gz   => 3383 bytes

[Eli Delventhal]

Added kzip shrinker, using pjt33's gzip=>gz code



A.java => 13931 bytes
A.class => 11196 bytes
A.normal.jar => 6846 bytes
A.progrd.jar => 4480 bytes
A.normal.pack => 5783 bytes
A.progrd.pack => 5750 bytes
A.normal.7z.pack.gz   => 3597 bytes
A.progrd.7z.pack.gz   => 3513 bytes
A.normal.kz.pack.gz   => 3480 bytes
A.progrd.kz.pack.gz   => 3383 bytes

Holy moly. Lemme try it out.

Wow! That works wonders! I can't believe how much more I can add now...

F.java => 22744 bytes
F.class => 11039 bytes
F.normal.jar => 7002 bytes
F.progrd.jar => 3843 bytes
F.normal.pack => 4776 bytes
F.progrd.pack => 4517 bytes
F.normal.7z.pack.gz   => 3177 bytes
F.progrd.7z.pack.gz   => 2997 bytes
F.normal.kz.pack.gz   => 3052 bytes
F.progrd.kz.pack.gz   => 2884 bytes

Hm, I clicked the bottom two links (the kz links) and I get:

FileNotFound: /home/indiespot/http-files/4k/64d01486f26c4116cf97b02a126867bd.4.pack.gz

The 7-zip links work, however.

[Riven]

I just made another update (thanks kapta) which reduces the size even more.

[Riven]

Hm, I clicked the bottom two links (the kz links) and I get:
The 7-zip links work, however.

Thanks, fixed it!

[SquashMonster]

Oh my, this is too good to be true.  My game just went from 4110 bytes to 3397.  Thank you!

I'm almost at a loss for what to do with all that space, haha.

[Riven]

A.java => 13931 bytes
A.class => 9627 bytes
A.normal.jar => 6266 bytes
A.progrd.jar => 3817 bytes
A.normal.pack => 4707 bytes
A.progrd.pack => 4430 bytes
A.normal.7z.pack.gz   => 3201 bytes
A.progrd.7z.pack.gz   => 3035 bytes
A.normal.kz.pack.gz   => 3077 bytes
A.progrd.kz.pack.gz   => 2932 bytes

[moogie]

That is some serious compression!

I have been attempting to replicate your results but so far have been unable to The closest i have come is 1244 bytes compared to your tool's 1226 bytes for my sample code.

My steps:

1. javac -target 1.5 a.java (version 1.6.0_12)

2. proguard (version 4.5 beta 2) with the following config file:

Code:

-libraryjars "D:\Program Files\Java\jre6\lib\rt.jar"
-injars "D:\temp\a.jar"
-outjars "D:\temp\a.proguard.jar"
-allowaccessmodification
-overloadaggressively
-optimizationpasses 99
-defaultpackage ''
-keep public class a extends java.applet.Applet

note that the JRE version it uses is  1.6.0_13-b03.

3. pack200 with the following settings

Code:

    p.put(Packer.EFFORT, "9");
    p.put(Packer.SEGMENT_LIMIT, "-1");
    p.put(Packer.KEEP_FILE_ORDER, Packer.FALSE);
    p.put(Packer.CODE_ATTRIBUTE_PFX+"LineNumberTable", Packer.STRIP);
    p.put(Packer.CODE_ATTRIBUTE_PFX+"LocalVariableTable", Packer.STRIP);
    p.put(Packer.CODE_ATTRIBUTE_PFX+"StackMapTable", Packer.STRIP);
    p.put(Packer.CODE_ATTRIBUTE_PFX+"SourceFile", Packer.STRIP);
    p.put(Packer.CODE_ATTRIBUTE_PFX+"Deprecated", Packer.STRIP);
    p.put(Packer.CODE_ATTRIBUTE_PFX+"Exceptions", Packer.STRIP);
    p.put(Packer.CLASS_ATTRIBUTE_PFX+"LineNumberTable", Packer.STRIP);
    p.put(Packer.CLASS_ATTRIBUTE_PFX+"LocalVariableTable", Packer.STRIP);
    p.put(Packer.CLASS_ATTRIBUTE_PFX+"StackMapTable", Packer.STRIP);
    p.put(Packer.CLASS_ATTRIBUTE_PFX+"SourceFile", Packer.STRIP);
    p.put(Packer.CLASS_ATTRIBUTE_PFX+"Deprecated", Packer.STRIP);
    p.put(Packer.CLASS_ATTRIBUTE_PFX+"Exceptions", Packer.STRIP);
    p.put(Packer.METHOD_ATTRIBUTE_PFX+"LineNumberTable", Packer.STRIP);
    p.put(Packer.METHOD_ATTRIBUTE_PFX+"LocalVariableTable", Packer.STRIP);
    p.put(Packer.METHOD_ATTRIBUTE_PFX+"StackMapTable", Packer.STRIP);
    p.put(Packer.METHOD_ATTRIBUTE_PFX+"SourceFile", Packer.STRIP);
    p.put(Packer.METHOD_ATTRIBUTE_PFX+"Deprecated", Packer.STRIP);
    p.put(Packer.METHOD_ATTRIBUTE_PFX+"Exceptions", Packer.STRIP);    
    p.put(Packer.MODIFICATION_TIME, Packer.LATEST);
    p.put(Packer.DEFLATE_HINT, Packer.FALSE);
    p.put(Packer.UNKNOWN_ATTRIBUTE, Packer.STRIP);

3. kzip the resultant pack file with the following command line option

Code:

kzip.exe /rn /q /y /b 40 ...."

found after a brute force search on block size.

4. using pjt33's zip to gz conversion code to produce the final result.

Are you doing anything different or using different versions?

[Riven]

I have been attempting to replicate your results but so far have been unable to The closest i have come is 1244 bytes compared to your tool's 1226 bytes for my sample code.

I'm actually doing much less.

For javac: -g:none -target 1.5
For proguard: -optimizationpasses 16 -keep public class *
For pack200: -effort 9 --strip-debug
For 7z: (nothing)
For kzip: -y -b{0,8,16,24,32,....,512}

Ofcourse I could say I added a lot of dirty tricks, but it looks like I'm not even trying, doesn't it?

[Eli Delventhal]

Hey Riven, do you know that your tool works for all Applets? Mine doesn't seem to run. I haven't actually messed with it for, well, a year, so it's certainly possible that something's actually broken in my code, I don't know. I'm just curious if other people have been able to run applets packed with your system (using the 7-zip proguard).

[Riven]

kevglass seems to be able to run his Emulator running Mario as an applet, after my shrinker did its job.

Keep in mind that for a *.pack.gz to be run as an applet, you have to request a *.jar from the server, at which time the webserver looks at the Accept-Encoding header, and returns the contents of a (potential) *.pack.gz file instead (thanks Sun, for making it so obvious)

[Eli Delventhal]

kevglass seems to be able to run his Emulator running Mario as an applet, after my shrinker did its job.

Keep in mind that for a *.pack.gz to be run as an applet, you have to request a *.jar from the server, at which time the webserver looks at the Accept-Encoding header, and returns the contents of a (potential) *.pack.gz file instead (thanks Sun, for making it so obvious)

Yeah. I submitted the file to Java4k and it doesn't run there, but assumedly Appel already has that webservice working. Looks like I need to dive back into that source code and see if I can remember what's going on...

[Riven]

Hey Riven, do you know that your tool works for all Applets? Mine doesn't seem to run. I haven't actually messed with it for, well, a year, so it's certainly possible that something's actually broken in my code

Then try t run it as a JAR-applet. If it works there, and the pack.gz doesn't work, let me know.

[Eli Delventhal]

Then try t run it as a JAR-applet. If it works there, and the pack.gz doesn't work, let me know.

Yeah, actually it was both applet-able and webstart-able and I just went and tried to compile/run it (using the main method) and all I got was a blank window. So it appears that I broke something somewhere along the way. Thanks.

[pjt33]

Hey Riven, do you know that your tool works for all Applets? Mine doesn't seem to run. I haven't actually messed with it for, well, a year, so it's certainly possible that something's actually broken in my code, I don't know. I'm just curious if other people have been able to run applets packed with your system (using the 7-zip proguard).

I'm not using Riven's system, but... I have a Proguard-pack200-kzip compilation script, and somewhere along the line it seems to be breaking the string which encodes my data. Once I work out what's going on there I should be able to upload my first game; my current suspicion is that Proguard is responsible.

Edit: hadn't updated the script and Proguard 4.3 was breaking it. Proguard 4.4 not only fixes that problem but seems to compress a *lot* better.

[moogie]

I'm actually doing much less.

For javac: -g:none -target 1.5
For proguard: -optimizationpasses 16 -keep public class *
For pack200: -effort 9 --strip-debug
For 7z: (nothing)
For kzip: -y -b{0,8,16,24,32,....,512}

Ofcourse I could say I added a lot of dirty tricks, but it looks like I'm not even trying, doesn't it?

i still think you have left out the call to doMagicStuff() in your steps as i have run my sample though the steps you list and have created slightly worse result than I had before (1247 instead of 1244, compared to your 1226)

The only thing i can think of is that we are using different versions of java and proguard and kzip. If it is not too much trouble can you tell me the versions you use? cheers!

[Riven]

The latest, downloaded them all over the last few days.

Well, except that I used JDK 6u15 instead of 6u17, and all software is running on Linux.

[zammbi]

A.java => 13931 bytes
A.class => 9627 bytes
A.normal.jar => 6266 bytes
A.progrd.jar => 3817 bytes
A.normal.pack => 4707 bytes
A.progrd.pack => 4430 bytes
A.normal.7z.pack.gz   => 3201 bytes
A.progrd.7z.pack.gz   => 3035 bytes
A.normal.kz.pack.gz   => 3077 bytes
A.progrd.kz.pack.gz   => 2932 bytes

Great stuff there. Now people can worry less on optimising.

Off topic.
Riven, I thought I was going slightly crazy when I saw your display pic. How does it get the username?

[Riven]

Great stuff there. Now people can worry less on optimising.

That indeed is the point. It might not result in the absolute smallest file, but it levels the playing field a bit.



Off topic.

Riven, I thought I was going slightly crazy when I saw your display pic. How does it get the username?

Congratz, you are the first to notice without being told (I guess). It's magic, and it's watching you.

[SimonH]

IP based...

[Riven]

IP based...

That's too easy.

[SimonH]

...but how to get the usernames?

*edit* HAS to be a lookup table!? If not - I give in! It is magic!

[zammbi]

Congratz, you are the first to notice without being told (I guess). It's magic, and it's watching you.

Well I was like wtf, did I really saw my user name in that image. So I right click and said show picture in a new tab. Then its url showed it as png, and then it really freaked me out (Though maybe chrome supports animated pngs now days too)
Then found out it was really a gif.
I guess it gets the username from the forum somehow, but not totally sure how would it.

[moogie]

Now people can worry less on optimising.

I find creating tools to get the most compression the most enjoyable aspect of the competition but some might call me strange!

[steveyO]

I find creating tools to get the most compression the most enjoyable aspect of the competition Smiley but some might call me strange!

Yep!  Thats strange.. Mind you, wont complain if you release version 4 

Wow.. So we have a magician at JGO.. Very impressive.. However after last refresh I am now pjt33 instead of steveyO.. Pretty cool nevertheless.. When I get 5mins break, gonna figure out how you do this..

[Riven]

Wow.. So we have a magician at JGO.. Very impressive.. However after last refresh I am now pjt33 instead of steveyO.. Pretty cool nevertheless.. When I get 5mins break, gonna figure out how you do this..

Heuristics do fail every once in a while.

Update: should fail less often now.

[zammbi]

I guess one way would be to keep looking at the who's online list when the image is pulled from the server. Enough polls would get it right.

[pjt33]

I guess one way would be to keep looking at the who's online list when the image is pulled from the server. Enough polls would get it right.

I was just thinking along similar lines.

[Riven]

I might be wrong, and I'm probably looking at the wrong code, but I think I'm using session hijacking and a zeroday security hole.

[pjt33]

I might be wrong, and I'm probably looking at the wrong code, but I think I'm using session hijacking and a zeroday security hole.

I had a look for JavaScript but I couldn't find any, so I assumed you weren't actually hijacking the session. Hmm.

Packet sniffing doesn't show an actual session hijack, but the referrer should allow you to see who's requesting the image.