Que sera, sera

I've recently been trying to understand the memory-allocation policy that OSX uses. I was getting 'vm_allocate failure' messages on the console, for one of my apps which is generally not a good thing. The app in question uses a *lot* of memory temporarily, over and over again. It can allocate and free large chunks of memory (say 100 to 200 MBytes at a time) depending on what the user is doing. "top" was telling me I was using about 1GB of RAM in the steady-state, and with 4GB in the machine, I wanted to know why I couldn't allocate another 200MB.

It turns out that I was suffering from memory-fragmentation. Even a relatively small request (anything more than a single page) will allocate from the set of 'large' memory regions for the application in its address space. The pairing of genuinely large allocations (the above mentioned 100-200 chunks) and smaller allocations meant that memory-regions weren't being freed back to the OS, even though I had called 'free' on the large chunk of RAM. As far as the OS was concerned, I'd freed my memory just fine, but the VM system still had me using the region because there were other allocations within that region.

This is counter-intuitive (but completely obvious when you think about it :-). Only memory-regions that are completely empty (ie: all allocated pages within the region have been freed) will be returned to the OS. Until that time, even though the OS will tell you that memory is available, it really isn't, until the memory-region in which that memory was allocated is completely empty.

I wrote a small tool to display the output of 'vmstat' (because I found it hard to visualise). If anyone wants it, I've made it available as a DMG containing the source project. You can specify the program to monitor, and either take snapshots or periodically sample it. The entire 4GB of address-space is shown as 1 page per pixel (so 1024x1024 pixels) starting in the lower left corner. Clicking or dragging the mouse over allocated pixels will give more details on the allocation.

Perhaps it'll be useful to someone else :-)

So, I've moved the blog over to RapidWeaver. Up until this point I've been using iWeb, but I was never really that happy with iWeb - it's a bit too limited for me.

I had actually been developing a web-creation system (and maybe I'll carry on) called 'Kaligraphy'. The idea was to take iWeb and make it more powerful, to make an application that was very integrated with a PHP engine on the web-server. The OSX-side was going to be just a layout-creation tool, with the text being flowed via a content-management system written in PHP.

I've written a content-management system before ... I wrote the one powering Incisive Media's websites, at last count there were over a hundred thousand webpages being served from that system. I wrote a full-text indexing system to run over it that was an order of magnitude faster than the MySQL built-in text-index. I wrote a mass-mailing engine (*) that could key off that search-engine to produce dynamic results and send a million or so emails per day with full read/response tracking. I wrote modules that allowed subscriptions by product, by page-count, by time, etc. What I am trying to say is that I have some experience in dynamic websites... I was hoping to bring that experience to bear while creating a graphical front-end on the Mac rather than the form-based front-end that we have right now.

To bring this post (kicking and screaming) back on topic - I've moved to RapidWeaver mainly because I have another 'big' project underway , and the fact that it has a plugin architecture. That ought to give me sufficient flexibility for what I want to do, without having to write a complete application to do it. The new idea is fantastic - even revolutionary :-) so I can't see myself finishing Kaligraphy in preference... So much to do, so little time...

I was considering going with SandVox, but with the 'MacHeist' offer available, I thought I'd try RapidWeaver instead. I've seen the pro's and cons argued back and forth (and to be honest, I'm not sure I'd have gone with the offer if I was a developer of one of the products), but from an end-user's perspective, I think it's a good-enough deal to go for. Personally it was the website-layout tool and one of the games (enigmo2) that made it worthwhile. I guess everyone will be different...

I guess the other point is that some poor deluded folk have actually subscribed to the RSS feed on here, and they're going to need to resubscribe to the new feed if they want to keep up-to-date.

(*) Incisive are good-guys - you only get mailed if you've genuinely signed up for something, and every mail contains a link that will actually unsubscribe you, rather than add your email to a "verified" list as some people do.

The next big stage of “project House” is now underway :-)

The image to the right is one of the reasons I bought this house in the first place - it has a huge back yard. Behind the garage, there’s this area the previous tenants used as a play-area for their kids, but from the get-go, this was marked down as where the swimming pool was going to go :-)

Basically I’m after what is known as a ‘lap pool’ in the US - long and thin. There’s a perimeter limit of 120’ after which you start paying extortionate amounts of cash, so it’s going to be 50’ by 10’.


So, on Monday the final measurements were taken, and they promised to be back on Wednesday to start digging. I had to take a half-day off work on both days, just to be there and give final approvals etc.

The digging crew turned up on Wednesday (6th Dec), and you can see what they’d managed to do by the end of the day. This photo (left) is actually from early on Thursday morning before I left for work - it’s too dark to snap photos at the end of the day. They’d cleared all the concrete slabs, and started on the main earth-removal for the pool.

Unfortunately they had to rip out the sprinkler-system control-wires (so that’ll have to be re-laid) and the internet-connection to the garage went along the same pipe conduit, so that’s gone too. I’ll have to get both of these reinstalled when the concrete gets laid.

I’ve also lost a fair amount of sod due to the heavy machinery they use while excavating, but I expected that.



By the end of Thursday (7th Dec), the pool was excavated properly, (photos immediately above) and it was really obvious what it was going to look like when it was all finished. They’s also dug out the hot-tub hole, which is not connected (again, for perimeter-reasons) but is only 2’ away from the deep-end. I’m looking forward to that :-) The pool ranges from 3.5’ deep at the shallow-end, to 6.5’ deep at the deep-end on a gradual slope.

The next thing that’s supposed to happen is for the lattice-work of iron supports to be installed. I was hoping that would be done on Friday, but they didn’t come. It started raining towards the end of Friday (and has continued throughout the weekend!) so perhaps that’s why. Regardless, I’ll be keeping this up-to-date online as things happen ...?

Can’t wait for it to be finished. The estimate is for late Jan, early Feb... :-)



So, it’s now the end of the week (15th Dec). The plumbers came on Monday 11th, and put in all the drains and piping, There’s a load of pipes sticking out - it looks as though they’re far too far out, but I’m guessing that’s just a safety margin. I think they’ll be trimmed once the gunite has been sprayed on.

The next day (12th Dec), the steel rebar work was done - you can see the mesh of steel that’ll supply structural strength to the concrete. This was milestone 1, so I had to cough up 25% of the cost at this point. It was time for the first county inspection as well - I’m assuming that went ok, I’ve not heard aything different...

I’ve got the cement contractor coming on Monday, to talk about the gunite process...

Monday came and went, but on Tuesday (20th Dec), the gunite crew turned up and started work on padding out the raw steel layout you can see in the above photos into a more-real swimming-pool and hot-tub structure.



This took them about 7 hours, starting at ~7:00am and finishing at ~2:00, at which point they left to do another pool... Take a look at the below photos to see what it looked like when they were making the walls, and a close-up on the tub - it's hard to get scale on the hot-tub, but from outer-wall to outer-wall there is 8' across :-)



The next things to happen were the coping being put on, the gas and electricity being installed, and the solar-power heating system being installed on the roof of the house... No real photos to show here - I'll post some up when the concrete is being laid...

So, now (Feb 5th) the coping has been laid down, the electric has been installed to the deck next to the pool, and the hardcore has been laid where the cement is going to go, with the rebar being placed, ready for the pour. We had the inspector out today, and she passed the pool ready for the pour - hopefully on Friday :-)

You can see lots of white pipes sticking up out of the hardcore - these are mainly drainage pipes (because there's a lot of cement coverage here), and you can just see (next to the fence) an extra drain I asked them to put in - in case I decide to put a shower out here to save traipsing into the house all the time...



I'm going for a brownish stamped-concrete look around the pool, which ought to go well with the burgundy coping. The next stage will take some time though - a couple of weeks for the concrete to cure before they come back to seal it. Waiting... Still waiting....

So, it's mid March, the concrete is down, and they've done a really nice job on the stamped finish, you can see where they've installed the pump and heater on the right-hand-side image below:



So, all that was left was to fill it with water... The larger images (click on a preview) look a lot better than these previews, there appears to be some aliasing on the previews, and running a blur-filter over the image doesn't really make it look better (different, but not better...). Anyway, first the pool from the perspective I've been using, and then from down low - that's what it looks like when you're swimming lengths...






and then the spa ... This is set to be at 102˚F when it's switched on :-) It's gorgeous :-)




If you look, you can see the 'Ray-vac' cleaning the bottom of the pool (it does this all day :-). It's strangely fascinating to watch this completely non-alive object running around at the bottom of the pool, climbing the walls, swishing its 'tail' after it. It looks really (no, really) alive...

The only thing left to do now is get the solar heating array linked up (it's on two sides of the roof of the house, the S and E slopes). This has to wait until the pump has run for a while, to get the first pass of all the gunk out of the pool (mainly gravel). The heater also needs a new gas-meter, it's a 400,000 BTU unit, and the (somewhat limited) gas supply can cause it to not start properly sometimes. That's in-progress too.

I also want to landscape the far rear of the garden, put in the shower/changing room where I had the drainage extended to, and then I can finally stop spending money on this :-) It's well worth it, though :-) It's taken about 3 months to get this far, and sometimes you wonder if it'll ever finish, but now that it is (more or less), I'm a really happy camper :-)