Software
The iphone and open-source (take 2)
25/07/08 07:38 |
Permalink
I'm really
getting tired of this...
Over at Aristotle Pagaltzis' blog he takes John Gruber to task for not understanding the GPL. Unfortunately it's Aristotle who has his argument confused.
As mentioned before all you have to do is distribute the source code. I've looked in the developer agreement, and there's nothing about what you have to do with the source code - it's all to do with the "Application" (a defined technical term that does not include the source code). A simple analogy to bread and flour ought to be sufficient to show that the source code is separate from the application itself...
So, here's what you do to fully and freely distribute open-source code.
If they want to run it on the phone itself, they'll need to pay (once) $99 to Apple to obtain the right-to-licence certificate-generation option. Now they can generate an ad-hoc certificate (as I mentioned in the previous post) and load the program they've just downloaded onto their own phone.
So, to go over the freedoms that this gives you, in Aristotle's own terms:
... looks as though John Gruber understands freedom better than Aristotle does...
Over at Aristotle Pagaltzis' blog he takes John Gruber to task for not understanding the GPL. Unfortunately it's Aristotle who has his argument confused.
As mentioned before all you have to do is distribute the source code. I've looked in the developer agreement, and there's nothing about what you have to do with the source code - it's all to do with the "Application" (a defined technical term that does not include the source code). A simple analogy to bread and flour ought to be sufficient to show that the source code is separate from the application itself...
So, here's what you do to fully and freely distribute open-source code.
- Set up
a website or get a project on one of the many
project-hosting sites
- Place a
tarball of your source code on that site and link
to it so people can download it
- That's
it
If they want to run it on the phone itself, they'll need to pay (once) $99 to Apple to obtain the right-to-licence certificate-generation option. Now they can generate an ad-hoc certificate (as I mentioned in the previous post) and load the program they've just downloaded onto their own phone.
So, to go over the freedoms that this gives you, in Aristotle's own terms:
- The
freedom to run the program for any purpose (freedom
0) check
- The
freedom to study how the program works, and adapt
it to your own needs (freedom 1) check
- The
freedom to redistribute copies to help your
neighbour (freedom 2) check
- The
freedom to improve the program, and release your
improvements to the public, so that the whole
community benefits (freedom 3) check
... looks as though John Gruber understands freedom better than Aristotle does...
The iphone and open-source
18/07/08 12:23 |
Permalink
The FSF have
just gone on a bit of a rant
about how
locked-up and closed-source the iphone is, which
caused a fair amount of commentary and
meta-commentary, but the problem is that the basic
tenet of the FSF's screed is just not true. The
very first point they "make" is:
Now I have recently got my personal iphone development environment set up. I downloaded the 'accelerometer' example source as a DMG from Apple (this app graphs the accelerometer in real-time) - there's nothing special about this source code, I just wanted something a *little* more complex than 'hello world'. Any other source-code distribution would have illustrated my point just as well.
So, I went through the various certificate-signing things, and created development, distribution, and ad-hoc certificates. I compiled the code and dragged my ad-hoc certificate and the application onto itunes, then synced with my phone.
The result is that I have some-random-program (in this case the accelerometer app) whose source-code I downloaded from the internet installed and running on my iphone. I did without jailbreaking, or doing anything non-official according to Apple. I need the ad-hoc certificate at *compile-time*, which authorises my iPhone to be able to run the app, but if you're distributing open-source code, that's just fine and peachy - any recipient will want to compile it themselves anyway.
So, here's the choices if you want to code open-source stuff:
The *only* barrier to #2, #3 is the cost of the developer program, ($99) which isn't much of a barrier. If $99 is truly out of your reach, you probably ought not be fixating on luxury items like the iphone - and in any case you can still download, compile, and run the code in the simulator. This is open-source in every aspect, and my respect for the FSF has gone down as a result of their campaign. It reminds me of Greenpeace, who don't really give a damn about whether Apple *are* green, they only care funding themselves and Apple are a high-profile target name.
"iPhone completely blocks free software. Developers must pay a tax to Apple, who becomes the sole authority over what can and can't be on everyone's phones."
Now I have recently got my personal iphone development environment set up. I downloaded the 'accelerometer' example source as a DMG from Apple (this app graphs the accelerometer in real-time) - there's nothing special about this source code, I just wanted something a *little* more complex than 'hello world'. Any other source-code distribution would have illustrated my point just as well.
So, I went through the various certificate-signing things, and created development, distribution, and ad-hoc certificates. I compiled the code and dragged my ad-hoc certificate and the application onto itunes, then synced with my phone.
The result is that I have some-random-program (in this case the accelerometer app) whose source-code I downloaded from the internet installed and running on my iphone. I did without jailbreaking, or doing anything non-official according to Apple. I need the ad-hoc certificate at *compile-time*, which authorises my iPhone to be able to run the app, but if you're distributing open-source code, that's just fine and peachy - any recipient will want to compile it themselves anyway.
So, here's the choices if you want to code open-source stuff:
- Generate an ad-hoc
certificate for a set of phones (max 100) and
deliver the certificate along with the app (binary
and source). You can distribute binaries like this
for an identified set of phones.
- Distribute your source
code. Developers can compile their own version of
the app and install onto their own phone using
their own ad-hoc certificates
- Distribute the
source-code on your website as above, and the
binary via the app-store (for free).
The *only* barrier to #2, #3 is the cost of the developer program, ($99) which isn't much of a barrier. If $99 is truly out of your reach, you probably ought not be fixating on luxury items like the iphone - and in any case you can still download, compile, and run the code in the simulator. This is open-source in every aspect, and my respect for the FSF has gone down as a result of their campaign. It reminds me of Greenpeace, who don't really give a damn about whether Apple *are* green, they only care funding themselves and Apple are a high-profile target name.
Networked SQLite
18/06/08 19:09 |
Permalink
I've been
playing around with SQLite for some time now, and one
of the things I thought I'd try out was a
client-server version of the database - so
here it
is. It's
a pretty basic implementation, but there was some
chatter on the sqlite mailing list, so I thought
I'd package it up and release it.
I originally wrote a lot of this in Objective C, with proper classes and methods. In an attempt to make it a bit more portable for those who haven't seen the light, I stepped down to using Foundation and plain-old-C, well apart from the demo client code anyway. ObjC is *so* much nicer than C I had to use it there :) You can get FoundationLite from Apple's website, and that should be all you need to compile against this source-code.
This is an earlier version of the code I used in a work project, so it has less functionality than the one I have at work, but I may start to fold some of that back in. I guess, as long as I re-implement rather than copy, I'm probably ok. Work are using a whole slew of my code anyway, so I reckon we're even on that score :)
The source is freeware - do with it as you please. If you'd like to credit me somewhere, I certainly wouldn't say no, but that's not necessary.
Run the server with 'sqld -I' to initalise a database in /opt/db, then do 'sqlc -u root -p sqld -d sqld' to connect to the server from a different terminal (there's -h server-hostname as well, if you want to connect via TCP). There's an auth table (initially set up as user=root, password=sqld, database=sqld) and a 'create database' pseudo-command to create new databases (which are just files in the install-dir (/opt/db by default).
This version doesn't have 'show tables', 'desc tablename' etc. The authorisation stuff does work, or at least it should. It doesn't have the change-management feature either (where different connections are notified asynchronously that cols X,Y,Z have changed in rows A,B,C in table T. I'll probably get around to putting those things back in. I think I had server-side plugins working as well - so pseudo-commands could be implemented ung loadable plugins.
It's all based around a simple packet-library concept which abstracts the network-transport. It looks a bit ugly in plain-old-C, but it does work reasonably well. Example packet dump from the server which corresponds to a client-connection trying to authenticate against the server (the three fields are 'username', 'password' and 'database')...
I originally wrote a lot of this in Objective C, with proper classes and methods. In an attempt to make it a bit more portable for those who haven't seen the light, I stepped down to using Foundation and plain-old-C, well apart from the demo client code anyway. ObjC is *so* much nicer than C I had to use it there :) You can get FoundationLite from Apple's website, and that should be all you need to compile against this source-code.
This is an earlier version of the code I used in a work project, so it has less functionality than the one I have at work, but I may start to fold some of that back in. I guess, as long as I re-implement rather than copy, I'm probably ok. Work are using a whole slew of my code anyway, so I reckon we're even on that score :)
The source is freeware - do with it as you please. If you'd like to credit me somewhere, I certainly wouldn't say no, but that's not necessary.
Run the server with 'sqld -I' to initalise a database in /opt/db, then do 'sqlc -u root -p sqld -d sqld' to connect to the server from a different terminal (there's -h server-hostname as well, if you want to connect via TCP). There's an auth table (initially set up as user=root, password=sqld, database=sqld) and a 'create database' pseudo-command to create new databases (which are just files in the install-dir (/opt/db by default).
This version doesn't have 'show tables', 'desc tablename' etc. The authorisation stuff does work, or at least it should. It doesn't have the change-management feature either (where different connections are notified asynchronously that cols X,Y,Z have changed in rows A,B,C in table T. I'll probably get around to putting those things back in. I think I had server-side plugins working as well - so pseudo-commands could be implemented ung loadable plugins.
It's all based around a simple packet-library concept which abstracts the network-transport. It looks a bit ugly in plain-old-C, but it does work reasonably well. Example packet dump from the server which corresponds to a client-connection trying to authenticate against the server (the three fields are 'username', 'password' and 'database')...
Address-space monitoring
17/12/06 23:04 |
Permalink
I've recently been trying to understand the
memory-allocation policy that OSX uses. I was getting
'vm_allocate failure' messages on the console, for
one of my apps which is generally not a good thing.
The app in question uses a *lot* of memory
temporarily, over and over again. It can allocate and
free large chunks of memory (say 100 to 200 MBytes at
a time) depending on what the user is doing. "top"
was telling me I was using about 1GB of RAM in the
steady-state, and with 4GB in the machine, I wanted
to know why I couldn't allocate another 200MB.
It turns out that I was suffering from memory-fragmentation. Even a relatively small request (anything more than a single page) will allocate from the set of 'large' memory regions for the application in its address space. The pairing of genuinely large allocations (the above mentioned 100-200 chunks) and smaller allocations meant that memory-regions weren't being freed back to the OS, even though I had called 'free' on the large chunk of RAM. As far as the OS was concerned, I'd freed my memory just fine, but the VM system still had me using the region because there were other allocations within that region.
This is counter-intuitive (but completely obvious when you think about it :-). Only memory-regions that are completely empty (ie: all allocated pages within the region have been freed) will be returned to the OS. Until that time, even though the OS will tell you that memory is available, it really isn't, until the memory-region in which that memory was allocated is completely empty.
I wrote a small tool to display the output of 'vmstat' (because I found it hard to visualise). If anyone wants it, I've made it available as a DMG containing the source project. You can specify the program to monitor, and either take snapshots or periodically sample it. The entire 4GB of address-space is shown as 1 page per pixel (so 1024x1024 pixels) starting in the lower left corner. Clicking or dragging the mouse over allocated pixels will give more details on the allocation.
Perhaps it'll be useful to someone else :-)
It turns out that I was suffering from memory-fragmentation. Even a relatively small request (anything more than a single page) will allocate from the set of 'large' memory regions for the application in its address space. The pairing of genuinely large allocations (the above mentioned 100-200 chunks) and smaller allocations meant that memory-regions weren't being freed back to the OS, even though I had called 'free' on the large chunk of RAM. As far as the OS was concerned, I'd freed my memory just fine, but the VM system still had me using the region because there were other allocations within that region.
This is counter-intuitive (but completely obvious when you think about it :-). Only memory-regions that are completely empty (ie: all allocated pages within the region have been freed) will be returned to the OS. Until that time, even though the OS will tell you that memory is available, it really isn't, until the memory-region in which that memory was allocated is completely empty.
I wrote a small tool to display the output of 'vmstat' (because I found it hard to visualise). If anyone wants it, I've made it available as a DMG containing the source project. You can specify the program to monitor, and either take snapshots or periodically sample it. The entire 4GB of address-space is shown as 1 page per pixel (so 1024x1024 pixels) starting in the lower left corner. Clicking or dragging the mouse over allocated pixels will give more details on the allocation.
Perhaps it'll be useful to someone else :-)
Wherefore art thou ?
15/08/06 21:09 |
Permalink
A while ago I wrote a small
web-app that performed geolocation on the
visitors IP address. I thought it would make a
decent journal entry [grin].
The basic premise was to use humans (when they visited) to correct the DB when it was wrong, and also to use robot searches (using the sarangworld regexps) to automate searches of IP addresses. More details at http://www.hostip.info/about.html
Hostip itself started off as a "wouldn't this be cool" idea, and a first version was born. The 'individual privacy' minded will have a field-day with this, but the inspiration actually came from watching 'Enemy of the State' on a '747 flight :-) I wanted to do (in a very limited way, of course) something similar using the web. As always in projects like this, it's the data that's the hard part of the equation, not the coding...
This first version allowed people to type in new cities, and it would auto-associate with their IP address. This was (as I should have forseen) a complete disaster. The number of Martians living here on Earth is truly amazing. We apparently even play host to a couple of Alpha Centaurites; to these fine beings I say 'Welcome to Earth' in "Will Smith" fashion. (Yes, I'm a fan...)
Once it was clear that if bad data was trivial to enter, it would indeed be entered, I raised the bar a little. Now you can only choose cities that already exist (and which I have latitude and longitude for), or email me with the details of a previously-unknown city, and I'll check it out before entering it into the DB. This has made the database more useful... Needless to say, cleansing all the bad data from the DB was a monumental task. It literally took weeks, and if I'd known at the start how long it would take, I'd not have started it!
It's still possible to lie to the machine of course (and I dare say lots do, on purpose, simply because it's their principle to do so). I have in my own way tried to get around that - the DB keeps a track history of assignments to a /24 netblock (that's the smallest unit it tracks), and since you can only reassign your own IP address, as soon as 2 others on your netblock tell the truth about where you are, it will switch to the real location... It's certainly not foolproof (hell I can think of a half-dozen ways around it!) but it raises the bar...
Up until this point, hostip was a purely text-based system. Next came the map data. I got in touch with the US National Geophysical Data Center in Boulder, and asked them for the highest-resolution data they had. That turned out to be 30 arc-second elevation data for the entire planet. Wow! So I spent some time writing tools to efficiently extract the correct data and colour it nicely/correctly for the small maps I needed - this took a week or so... Just loading the data into RAM took a lot of time (eventually I remembered mmap() and things went a *lot* smoother!).
The dataset consists of a 43000x21500 image, at approximately 1km/pixel, taking ~2.6Gbytes to store. Even things like ppmtogif can't handle that much data :-( The current database size (from du -sk on the mysql db directory) is 623Mbytes. All this needed to be correlated together before the applet started to look even vaguely reasonable. It still has lots of errors (mostly where I have the decimal point wrong in latitude or longitude figures :-( but it's useful now, and I tend to get told [grin] when something is wrong...
One of the reasons I wanted to do this (apart from the obvious coolness of the idea :-) is to give something back to the people who've given me so much 'free' software over the years. Those from this nameless multitude, I salute you - I hope you get as much out of hostip as I got from your various projects/programs.
I happen to think the applet is (even though I wrote it myself, [grin]) one of the coolest ones I've seen so far, although that may be down to knowing just how much hard work went into making it [big grin].