ducktyped

Many websites are composed of several fairly independent apps that would still benefit from tighter integration than running them as separate apps. Here's a typical site: core site, forum, blog/cms (or photo management app, etc).

The typical Rails response to multi-app integration is one of the following two: 1. "Just run them as separate apps on different subdomains and share the database." or 2. "Use the app as the base and add your own code to it." I have integrated several Rails apps, and it is dirty and inefficient, a real pain in the ass, especially compared to the joy of doing most other things with Ruby/Rails. The failure of multi-app Rails is documented by the plethora of hacks used to emulate multi-app behaviors: generators, engines, appable plugins, and the multi-app routing plugin.

The point of this post is not to rip on rails, it's just what I'm primarily using, and I would love to see other Ruby web frameworks such as merb, ramaze, and the micro-frameworks think about multi-app while they can still make major changes. Ruby is powerful enough to elegantly enable multi-app configurations.

What sucks about multi-app integration right now

Redundant css and js

You send a new copy of prototype.js when switching between your core app, beast, and mephisto for example. You duplicate CSS as well. To avoid that you end up futzing around with deployment managers (capistrano) to make one app use the other's css and js in production.

Duplicate models

You have to duplicate models to access resources across the separate apps, or set up SCM, symlink hacks for versioning. Ugly, and suckage for dev / deployment differences.

Shitty routing

You end up using hacks and the web server instead of the app's router to resolve your routing. It's less powerful and in the wrong place. Now there is the multi-app routing plugin, which is just a hack to mitigate an underlying bigger problem. (It's cool for some use cases, sure, but still limited in capabilities)

Code scales poorly

Dumping all apps' M/V/Cs into one big app creates a cluttered project, and the project code doesn't scale. Here's a post by caboose's Courtenay hinting at that the rails directory structure struggles with large apps. Having "eighty controllers" should not be a reason for having to split a project into two completely separate projects, maintaining separate deployments, etc.

In a way, the Rails folder structure is backwards, e.g. I can namespace my M/V/Cs but that means I have pieces of a semantically related structure (an app) in 3 places instead of one. When I work on code, I mostly work on a subcomponent of the app, and not just in M or V or C.

What I want is:

/app1
  /model
  /view
  /controller
/app2
  /model
  /view
  /controller

Not:

/model
  /app1
  /app2
/view
  /app1
  /app2
/controller
  /app1
  /app2

Memory overhead

People on shared hosts and small VPSes don't have ram to throw around. Right now, with a rails site consisting of a core app, blog, and forum, I get 3 hefty rails processes, when I want just one. (I could then use two mongrels, save 1/3 ram and get better concurrency/basic load balancing (if one app gets hit much more than the other)).

Single sign-on and session/cookie sharing

This isn't too hard to solve in practice, but it's not necessary. There should be a auth/login app to drop in instead of the generator. Other apps then by convention use user and user_id as the hooks into the auth/login system, but they don't all duplicate it. The auth/login system can be whatever, as long as the interface to it is a user model and user_id as the hook. All that would be left to do to integrate it is to add associations and before_filters. This has been a big topic of debate in the beast forums, and frankly, it shouldn't be that integrating a rails app with beast (another rails app) is exactly as cumbersome as integrating it with a PHP forum engine.

Fighting objections - it's all about views

Plugins already provide (80% of) this functionality

No they don't, and they shouldn't. Engines, appable plugins, and generators are all trying to fix part but not the root of the multi-app problem.

Most importantly, plugins have no views. Have your ever counted your LOC? Views make up a lot of code, especially with ajax thrown in. For example, I want to be able to drop in a blog or comment system that includes ajax posting of comments by default, rather than me having to write all the views from scratch. Plugins are fantastic for what they're designed for, but multi-app is not their use case.

High level components don't work

DHH is a declared opponent of components (or at least was, given that the quoted statements are over 2 years old). (see here and here).

His experience is based on a failing high level parameterized component architecture in java. But the components he's talking about are different from multi-app. I'm not saying that you can just drop in a bunch of components and wire them together with a few parameters. I'm saying, get canonical code (e.g. a basic blog engine) and adapt it to your needs in a way that's cleaner and more maintainable/reusable than a generator's output or nested plugin stuff like engines or appable plugins.

Also, David's experience was based on working in a java shop, but Ruby is much more concise than Java, and modifying a forum engine in Ruby isn't too bad, especially if its code layout follows a canonical layout such as the standard rails app structure.

Ruby/Rails makes it so easy to write from scratch, why bother?

Why would you need this stuff, if Ruby/Rails make it so easy to create a blog engine from scratch. DHH did it in 10 minutes in his video, Akita in 20 minutes. Did they really?

It's like saying "I can write a digg clone in 20 minutes." You can write something that provides the most basic functionality, but it's not a digg clone. You know why? UI, UI, UI. User interface is 80% of the value of an app for the end user, and right now there is no reuse of great UI design, which is a shame, because UI design is hard, and a lot of people would benefit from leveraging great UIs. I'd rather drop a mephisto-lite into my project than rewrite it from scratch to "meet my app specific needs". And I want to do it without mixing its code and my own apps code wildly all over the place (which just creates SCM headaches).

What could multi-app look like?

The one framework I found that gets multi-app mostly right is Python's django. It's great to be able to drop in a basic blog or forum engine that includes views/ajax, etc.

At the very least, it makes utter sense to have common routing, css, javascript for a collection of apps in a site.

DRY up the suckage described above

• CSS, JS, models
• Sessions/cookies
• Routing
• (Single sign-on)

Rough idea for a project/app layout

/app_root
    /config (global configuration, affects all apps)
    /public
        /js (main app default)
        /css 
        /blog/js (blog specific)
        /blog/css
        /forum/js
        /forum/css
    # These are the apps that would be dropped into the application
    # Test and spec folders would be moved into the app folders, so
    # that each app can have its own test suite
    /app (The default main app)
    /blog (Full app with tests, etc., minus config, public)
    /forum (same thing)

How to adjust routing

Routing could simply add a an 'app' level :app/:controller/:action/:id => http://mysite.com/forum/post/view/1

Problems to address

Layouts and css

How to avoid layout duplication, and where to put app-specific css. I would put css in public/css/#{app_name}/default.css, and then for deployment join the css files. This would also require some name-spacing conventions of css to avoid collisions/overrides. I haven't fully thought through how to do a DRY layout structure.

DB table name collisions

I like django's solution of prefixing tables with the app name, e.g. a posts model in a blog and a forum app would map to the following tables:

• blog_posts
• forum_posts

This could be easily done by using the ORMs' table prefix methods, and could be automated, dealt with by convention.

Different apps being tied to different versions of the framework

What if my core app runs on edge, but my dropped-in mephisto-lite relies on Rails 1.2.x. Since I'm now running within the same process, this wouldn't be possible AFAIK, but the tradeoff is still worth it. Especially if you reuse a lot of your own internal apps. It might even lead to better maintained apps.

I made the assumption of everything running in one process before, but that wouldn't necessarily have to be the case. Apps could run in their own processes with their own version of the framework, but I still have a smart wrapper around them for routing, etc.

Apps use different ORMs or databases

I haven't thought through this one too much, but it's even conceivable that apps could use different ORMs to talk to the database, or even use separate databases. I would probably have a global default db config, with possibilities for apps to override them. That might make the framework startup process overly complex, so I'd even be fine if all apps would have to use the same ORM/db. (Side note: Multi-db is still another unsolved challenge for most Ruby web frameworks).

Some folder locations might be contentious

Should /lib be global or per-app? Per-app keeps things more in one place and allows different apps to use different versions of the same library if necessary. Perhaps a structure of /lib/app_name would solve this in an elegant way.

Conclusion

There is movement in the Ruby web framework space with new frameworks hitting the scene. "The balls are in the air", and now is the time to bring up ideas for change. Rails has been and still is a fantastic yardstick by which to measure web frameworks, but Rails has warts too, and sometimes Rails is starting to feel a bit pudgy, like a "Ruby Struts" (which is amazing, because Rails is at least an order of magnitude better than Java Struts, it just shows how much progress is being made.)

I talked about multi-app in this post based on my personal experience with Rails. Rails is fantastic, but I don't think they would incorporate a clean multi-app setup, but I'm hopeful that the new kids on the Ruby web framework block (like merb, ramaze, sinatra, et al.) will think hard about if and how to incorporate multi-app capabilities. Especially some of the micro-frameworks could scale very nicely (code management and deployment-wise) by clustering small, light apps together (like a bliki for example).

I hope to stimulate a healthy, productive, and civilized discussion with this post. Hopefully, at least one framework will incorporate multi-app capabilities. Whichever Ruby web framework will end up with a clean multi-app setup will be the one I'll be using.