So response to my previous post has been fairly positive, despite not achieving the critical mass necessary to land on Daring Fireball— come on @gruber, pay attention ;o)
I’ve had a few conversations with people over the last week about some of the implementation details and suchlike, and I wanted to expound on the original idea a little. The first post went into some reasonable detail on the security and verification systems, but not so much on the type of processing supported, and how it would be setup in a way beneficial to the goals of both Apple and us developers (or ‘plebs’).
There are a few main areas where background processing of some kind is highly beneficial, for which I will detail some examples here. Maybe I’ll even get the chance to do a third post with some code involved somewhere…
This isn’t necessarily what most people would put first, but it’s hard to underestimate just how much of your live data needs can be handled by these magic little packets at once.
Manual Network Updates
Sometimes push notifications aren’t feasible to implement server-side, or can’t provide enough detailed information (case in point: Outpost). For such applications, the ability to perform some networking on a scheduled basis in the background would be a boon to the applications’ usability.
If you’re writing a to-do list app, or some form of calendar, you’ll probably want to alert the user to certain conditions even when your application isn’t running. Currently the only way to do this is to have a server which manages such notifications and uses APSN to pop up an alert on the user’s iPhone, but the costs for maintaining that service can be prohibitive.
If your application deals with large data-sets, it might benefit from performing some house-keeping tasks now and then, for instance database compaction. This is an ideal candidate for a task to run while the phone is locked.
Let’s look at each of these in some detail, so we can see how the setup would work, and how the system would enforce its policies.
Potential User: Facebook
Your application receives push notifications for new messages as they arrive, but it can’t do anything with that message unless the user chooses the ‘view’ option. If the user cancels the alert, then the application won’t find out about the new message until it does its own networking to fetch the details. A better approach would be to register a background daemon to handle the notifications, thus being able to store them directly into the application’s data store in some manner.
The daemon in this case would run in a no-networking sandbox, since that should be unnecessary for this use case, but would be able to make changes to the disk, subject to the same restrictions as the main application. Once it’s finished processing the incoming notification it can then prompt a standard system response/alert by exiting and returning a special result code which would be monitored by the background processor launchd instance. This way the application can cause the usual alert message to appear, an icon badge to be applied, etc, or it can suppress any standard APSN response action completely.
Manual Network Updates
Potential user: Outpost
In some cases—particularly if an app is based off a third-party service such as Basecamp—it is impossible or impractical to implement push notifications on the server-side, so the application has to do all the legwork at runtime. This can lead to poor user experience issues as the application has to stop and perform a lot of work at startup, when the user wants to just fire it up and check something quickly.
A solution to this would be to register a network download daemon. This would be registered using two key pieces of information: the interval at which it wants to run, and the details of the network resource it wants to access. The system would then tweak the launch times to attempt to coalesce networking operations, meaning that the network hardware would be powered up exactly once, by the system, prior to any such daemons running. The system could also open network connections ready for the daemons, possibly connecting it up to the daemon’s standard input and output streams, further allowing the system to make the major decisions on power and resource management. In this case, the daemon itself might be prohibited from making its own network connections; an alternative should be available however where a daemon could do so, though possibly with a limit on the number of such connections it could open at one time or in a single invocation.
The daemon would be free to do whatever it wished with the data it receives from the network. Those with larger data sets would perhaps simply drop the downloaded data somewhere where the main application could parse it upon launch, perhaps with enough minimal parsing to determine a suitable application icon badge value. Applications with smaller data sets could perform all the parsing internally, updating the application’s main data store directly.
The system would put strict limits on this type of daemon— it can monitor its run time and the amount of CPU and network resources it uses, potentially throttling it or telling it to quit outright once certain limits have been reached.
The system would provide some APIs for this daemon to provide data for an APSN-style alert or other notification to be presented to the user. These APIs would be used to send the data to the parent launchd process, and the application would return a specific result code to inform the service which form of alert (badge, text, sound) to perform after the daemon terminates.
Potential user: any to-do list or calendar-based application
Many applications on the App Store now maintain to-do lists or link to online calendar or time-management services. Similar to the manual network type above, it might not be possible or practical for the application developer to build APSN functionality into the service on the server-side, so some means of firing off notifications while the application isn’t running would be necessary to provide full value to your customers.
This type of daemon would be initially registered without any specific launch conditions; these conditions would be added later on, giving specific times at which it would launch. The daemon itself would be expected to only read from the application’s local data store, so it would run in a sandbox with no networking or disk write access, just read access to the application’s data store folders.
This item would in fact be a good candidate for a plugin-style API, similar to those used by Spotlight or QuickLook on the Macintosh. The plugin would be loaded, asked for data, and unloaded when done, and as such wouldn’t require the creation of many separate processes. This might make security and sandboxing a little more problematic to implement however; the best solution would be to spawn a sandboxed loader process for each plugin, but that still requires many of those to be spawned, leaving us only a little better off than using custom daemon processes outright.
Potential user: OmniFocus
Some applications manage fairly large and complicated data sets; databases in particular are in high demand on the iPhone due to their memory-light access to large amounts of structured data. Every now and then, however, these large data sets might need a little cleaning and pruning. This might be a more involved operation than you want to do while the user is interacting with your application, or it might be something which is good to do frequently (more frequently than your user firing up your app, especially if you’re using any of the other background processing types described above to access your data store).
This type of daemon would be registered with a launch interval specified in days. It would have no access to the network (it’s only working with a local data set after all) but would have read and write access to the main application’s sandbox. The exact launch time would be unspecified, and manageable by the system (to better coalesce multiple such processes together), but would in all likelihood be confined to times when the system is locked, preferably using a heuristic to determine the likelihood of it remaining locked for a good while.
The processes would be monitored for power usage carefully; of all the above, these are perhaps the most likely to cause drain on the system by continually moving data into and out of memory and accessing larger than usual parts of permanent storage. They would also need to have some form of IPC channel available to communicate with the main application so that they can finish up quickly in the event that the application is launched while the daemon is performing its duties. The app would need to prompt the daemon to finish up, then wait for a response back saying that the data store is safe to use. Such interrupted work might then be recommenced when the main application terminates, although that would likely be a function provided and performed by the background launchd instance itself rather than prompted explicitly by the application.
For the IPC, using an Apple-defined ObjC protocol with Distributed Objects would make the most sense, but at present the required runtime support isn’t included in the iPhone’s Foundation framework. An alternative would be a set of Mach IPC methods (defined using MIG) for the application to talk to its daemon via the launchd process itself. The application would talk to launchd, and launchd would pass on messages to the daemon, returning a safe result once the daemon has cleaned up and terminated.
The four scenarios above should cover most, if not all, common uses of background processing, while still allowing the system to carefully manage the use of its resources and to optimize such behaviour.
I had planned to push these ideas internally once I started my job at Apple, but since the INS decided to make me its bitch last week, I’m going to have to rely on good old-fashioned social networking to do the job. Perhaps some nice person on the iPhone team will read this and start showing it to the right people. That would be nice. It would help greatly to have it excessively retweeted too— the more people see and advocate such ideas, the more likely they are to materialize.
Now GO TO IT PEOPLE CHOP CHOP kthxbye ;o)