Boris Mann's blog

Facebook apps with Drupal

Fri, 31 Aug 2007 13:39:45 -0700

Last night we gathered at the Bryght office for our monthly Vancouver League of Drupaleers user group meeting. Theoretically I help run the group, but Dale McGladdery does most of the heavy lifting.

This month Dale got the guys from the Donat Group to come over and talk about the set of Facebook API modules that they are building to make it easier to build Facebook apps with Drupal.

All I can say is....wow. James and Ross took us through how they built their Opus Player Facebook app, and we had lots of discussion / beta feedback. They've built something very interesting, and I can't wait to see it hit Facebook.

After mainly looking at the app itself (related but not yet integrated into Project Opus, although those are the future plans -- to mirror the functionality in Facebook on the public Project Opus site as well), we dove right into some code. Not yet available in Drupal CVS (soon!), there is some code posted here.

So what does this mean? Well, the Donat Group team built a great Facebook app, and just happened to implement it using Drupal. Along the way (and, um, at least one rewrite / refactor :P) they built a great set of base tools for interacting with Facebook APIs and generating FBML content. The Drupal framework gets another win for being a great flexible base to build all sorts of functionality on top of.

Joyent meetup and thoughts on virtualization

Wed, 01 Aug 2007 08:53:33 -0700

Last night Mark Mayo hosted the first Joyent meetup in Vancouver.

I've been following the Joyent adventures for quite some time. They're a small company, but they're aiming high. A while back, they merged with TextDrive, and became a vertically integrated hosting + applications/product company (and even open-sourced their Rails-based Connector suite -- see you at OSCMS next year!). In many ways, this is what Bryght's partnership with WorkHabit has been about: if we hadn't found such great partners, we would have had to build their expertise as part of our own company. Application hosting is still something that isn't that well understood, and the commodity-shared-hosting-optimized-for-Dreamweaver is in for a rude awakening one of these days.

Still Bryght

Sun, 29 Jul 2007 21:56:44 -0700

Yes, it's true, James is moving on from the Bryght team. What's the scoop? As James says, not much to gossip about. It was always tough to be the guy out in the 'burbs half a continent away. True, there's our favorite African, Adrian, but we're working on getting him moved to Vancouver as well.

I have very much enjoyed having James as a partner: thanks. Best of luck in your future adventures. And yes, I imagine Drupal will have a thing or two to do with that.

Drupal gets 20 Google Summer of Code projects for 2007

Thu, 12 Apr 2007 00:06:07 -0700

Wow. 20 projects accepted for Drupal for the 2007 version of Google Summer of Code. Thanks to all the mentors and especially top GSoC wranglers Robert Douglass and Angie Byron for herding us through this application process.

Not only did this pay off in 20 projects, but the folks that help host drupal.org at the Oregon State University Open Source Lab *also* got two projects accepted which are actually working with Drupal: integrating the Google API into Drupal.

There are some really great projects planned out. Here are some of my favourites:

RSS / Atom integration: I spent an hour or two with some of the guys from Achieve Internet last week on scoping some patches for the core aggregator code...there is lots of work to be done here, and I think we can get far, fast, by doing a bit of coordination
Jabber / XMPP: yay! my two favourite communities continue to collide
Scalability, Load Balancing, and High Availability: Vancouver local hadsie got in, which is great; Scott, want to work out of the Bryght offices for the summer?
SMS Framework: it's clear that mobile users are continuing to become first class citizens of the web, but that SMS is a great tool for easily integrating into virtually any mobile environment; can I hear the phrase "Twitter clone"?

Reverse Bounty: backport configurable watchdog logging to Drupal 5

Thu, 05 Apr 2007 16:29:35 -0700

It's been a while since I helped organize a reverse bounty. On the auction block is some great functionality that is likely going to make it's way into Drupal 6, but is super useful for:

high traffic sites
ISPs / mass hosting
enterprise deployments
people that like having their website instant message them

Today, the watchdog table logs regular system events (e.g. users logging in, creating new accounts, or posting content) as well as error conditions and other explanatory notes. On a high traffic site, the watchdog table must be pruned on a regular basis so that the database table doesn't grow wildly out of control. There is also no way to archive this information.

Enter Khalid's (2bits.com) patch for Drupal 6: instead of only logging to the database, the patch will enable the destination of logging to be configurable. Raw text file, syslog, and email logging are all modules that have been implemented as proof of concept using this new system.

But seeing this, many of us immediately wanted this functionality for Drupal 5. Much like Bryght's system included free tagging of categories and configurable RSS feed support in our hosted 4.6 version, functions in future versions of Drupal can be backported. That's what this reverse bounty is for – we've spoken with Khalid, and he's willing to make this functionality available via a patch to Drupal 5, including the porting of the support modules such as syslog and email notification. How can you help? Well, we're gathering funds using ChipIn -- click on the Flash widget below to contribute. If you really want to help promote, grab the ChipIn widget and post it on your own blog or website.

Cleanly overriding core modules in Drupal through the magic of multisite

Mon, 02 Apr 2007 17:14:38 -0700

I recently got an email from Mark Shropshire (apparently emailing from the depths of a secret research lab at UNCC) asking about an article I had written giving feedback on one of IBM's Developerworks articles, where I talked a bit about some best practices around multisite, symlinks, etc.

The bit that got him interested was this paragraph:

Overriding a core module is no problem. Any core module you place in /var/www/html/sites/domain.com/modules will override the main install core module. This is a safe way to have patched core modules without having to wonder if you'll run into trouble updating your main Drupal install.

Yes, it's true. With the power of Drupal's magic multisite file path scanning, you can place a module in your local sites/domain.com/modules directory and it will override modules that are "higher" up the search path. But, Drupal tries to thwart you by sneakily caching the file path to modules in the system table, which only refreshes when you go to the Admin / Modules page. Here's how to get that "local" module enabled instead:

0. Put your site in offline mode
1. Go to admin/modules (4.7 and earlier) or admin/build/modules (5.0 and up)
2. *Disable* the core module you want to override and submit the page (you should already have the override module in place)
3. Re-enable the same module, and you should now be running the "overridden" version

Google Apps now extensible: open source connections?

Thu, 22 Feb 2007 16:23:11 -0800

We've long been recommending Google Apps for your domain as a great email and calendaring solution. Further integration with Google's APIs has been "on the list", and now that Google has announced their Premier Edition, it looks like they're rolling out official ways to extend and integrate with the Google platform.

The Premier Edition is pricey (not expensive: it's in line with costs for big, Enterprise scale applications and support) at $50US/user account/year. How can open source work with this system? Can only the Premier Edition be extended with partner solutions? We'll be following this quite closely and looking at how our favourite open source project, Drupal, can work with these new Google features.

I've written a bit more about this topic over at my personal blog.

Another week, another release, breaking my head on install profiles

Thu, 25 Jan 2007 16:02:38 -0800

This year is going to be a bumpy one, but we're all pulling together and doing some great stuff. We just pushed another release of our bryghtbase codebase to our Bryght Light mass hosting service. It's the Drupal 5 release you get when you push signup, or the default install when you have one of our production hosting virtual private servers.

I've been breaking my head on install profiles. We naturally want to set up a bunch of stuff "out of the box". Now that Drupal 5 has "native" install profiles, we can work with the codebase directly, which makes things a lot easier. But, some things are still too hard. Basically, we need to make a library of CRUD functions for Drupal. Some of these may make it into future releases, ideally. For now, I've started scavenging some of the custom functions we've written in the past (e.g. one loops through profile fields and creates them -- see https://svn.bryght.com/dev/browser/bryghtbase/DRUPAL-5/trunk/html/profiles/crud.inc), and will be working with the community to standardize them.

Next up: planning for a Community profile and thinking about what we might put in a ProBlogger one.

OpenID Mashpit Vancouver, OpenID 2 code available for Drupal

Thu, 18 Jan 2007 14:01:19 -0800

Last night, a bunch of us Bryght guys attended the OpenID Mashpit hosted here in Vancouver by the folks at SXIP. It was a chance for a bunch of developer-types and interested techies to get together and dive into what, exactly, OpenID is and what this identity space is all about.

Dick Hardt started us off with an overview, so everyone was up to speed on the basic terminology and flow. The questions started coming pretty quickly and it turned into a very interactive session, proving that there is a lot of interest. We had a short break and then went into "lightning talks"....which ended up being more, longer interactive sessions.

I wanted to have an open session on "federation" – a word we never ended up defining. In short, in the network of networks world that is emerging, where any system/node can be both a consumer and provider of identity or attributes, how do pieces of the network easily share trust? The short answer is that trust is hard, but the OpenID 2 infrastructure likely gives us the extensible pieces we need (along with SAML payloads) to build such a system. We'll be focusing on making this work "out of the box" over the coming months. I had a great time during this session – I got up to the front and sort of set the stage, and then Dick Hardt joined me and we went back and forth, diving into the details of attribute exchange and how various parties in a system would grant or gain access to resources, attributes, etc. Thanks, Dick, let's take our show on the road!

When we moved into break out room, I went around the room and cornered people into installing plugins for different systems – we captured this on the wiki, and should be able to test logging into people's sites with our own OpenID URLs. For testing with OpenID 2 code, we found that SXIP has Email Verification and Attribute Exchange systems set up.

Oh yes, and as Steve Jobs likes to say....just one more thing: OpenID 2 code is now available for Drupal, with both consumer and server modules available. As far as we know, this is some of the first OpenID code out in the wild. James demo'd it last night, and Bryght has set up an identity provider at home.bryght.com. Feel free to use that as a home site, for testing or other purposes (the OSCMS Summit site has the compatible consumer module enabled). The code is not quite "finished" as James complains, but we wanted to get it out so that other people could start working on it with us and that we would be able to start testing. The big thing to note is that the OpenID 1.1 compatibility part isn't implemented yet, and large chunks of the attribute exchange aren't either.

Birthdays and releases, more tomorrow

Mon, 15 Jan 2007 23:39:42 -0800

As has been seen all over the web, Drupal turns 6 years old today, and is celebrating not only with cookies, but also with the release of Drupal 5.

Uwe Hermann was the first person I saw mention that Drupal shares this birthday with Wikipedia. Very cool!

Of course, at Bryght, we've been heavily involved in many things around this release, both as part of the community as well as getting many of our systems here prepped to do some new things. As I talked about in my post on mass hosting, Drupal 5 features install profiles: something that we envisioned three years ago and is the concept around which we built our Hostmaster code.

Our Hostmaster code now supports Drupal 5 install profiles natively. This means that any install profile created for a "regular" Drupal install can be mass hosted. It also means that we can build "native" profiles once using the rich variety of modules available, and then offer them point-and-click style in shared hosting, or make them available to VPS customers via SVN. And by VPS customers, I mean anyone -- you can kick the tires on our (very) Basic profile that we've put together as a Drupal 5 Preview Release (I wrote a bit more on groups.drupal.org). You'll see some more news on that from us tomorrow, when we "officially" bring all the Drupal 5 stuff live.

Virtualization and you

Wed, 03 Jan 2007 00:29:16 -0800

My post on mass hosting actually had a related topic that I ended up putting in draft mode because it got to be too long: how virtualization ties into hosting these days, especially as it relates to web apps.

We announced Bryght Hosting a while back, with just a few details. With our infrastructure partners at Firebright, we're going to make more bundles of services available based around a combination of the Xen virtualization platform and some new management tools that Firebright is developing. Lack of those tools is one of the reasons that IBM mentioned in their hosting and deployment article, where they describe using VMWare. A large part of these tools from Firebright (codename Rack Fire) will be open source and available to anyone interested in managing many virtual instances...but I digress -- go bug Firebright about that, their new site should be up in a couple of weeks.

IBM's write up is quite good, covering some of the basics around virtualization and why you would want to choose that. In the bad old days, a lot of virtual servers were "soft" virtual servers, where resources on a physical system were oversold. Typical Xen environments are "hard" virtual servers -- even if an environment doesn't use all the memory, disk space, etc. allocated to it, those resources are still fully allocated. Much more like really having your "own" server.

One of the first benefits is reliability. Say you're sitting on a piece of real hardware. A dedicated server. And the hardware goes bad. What happens next? Scrambling for backups and hoping for a new piece of hardware available. With virtualization (caveat: there are many ways to do this, and I'm handwaving around bits....), you can take the image of your entire environment, and move it over to another running Xen host and voila! you're still up. Backing up those server images, or even doing a trial upgrade on a complete clone of your production system -- all possible with a virtualized infrastructure.

Another benefit is scaling. Kris Buytaert (no relation to Dries Buytaert, although he is a friend) pokes some holes in the IBM choice of VMWare -- he likes Xen better, but also points to issues with scaling. I suspect that Kris tends to work with REALLY large deployments, and poking around further in his Automating Virtual Machine Deployment paper, it sounds like it is more focused on server consolidation than offer lots of small(er) hosted environments to lots of different people. He has some really good points about how you need to manage large scale deployments around updates between machines. Process becomes really important, so having a tuned, best practices environment where your processes are documented is very important.

So, virtualization and scaling. First off, you can start your site off with a little corner of a big machine. If traffic grows (and, one would hope, revenue) you can increase the size of your virtual server just by upgrading to a bigger piece of that machine. This takes maybe 5 minutes at most -- you "shut down" your virtual environment, it's adjusted to have more memory/diskspace, and then you bring it online again. No moving between servers, and pretty painless.

Of course, this only "scales" as big as the piece of physical hardware you're on. I'm not going to get into grid systems right now, but some of the ideas there are that you can scale incrementally, buying capacity as you need. We're cooking up something similar with Firebright: virtual clusters. For customers that want increased redundancy and/or future scalability, rather than starting on a small, single instance with Web + DB on one box, we can deploy a small cluster of virtual instances --- maybe 2 web front ends and a database backend, or an out-of-the-box redundant setup, with 3 web front ends and 2 database backends. Need to scale? First, the individual components can be allocated more resources -- from small web and db to medium web and db, etc. Next, since the whole setup has been configured to scale horizontally, you can spin up additional web and database nodes as needed. Caught in a Digg-storm? Add a few virtual environments temporarily, and then turn them off again when the traffic levels go back to normal.

Multisite and Mass hosting Drupal

Thu, 21 Dec 2006 10:36:24 -0800

Bryght started with a simple idea: what if we could take a powerful, complex framework like Drupal and make it available to everyone... with or without technical expertise. This was closely related to our belief that eventually static HTML pages on the Internet will be replaced by dynamic pages. Dynamic pages means web applications.

We've seen a lot of this occur over the past several years. When we started, Drupal was making the transition from Drupal 4.4 to 4.5. Multisite was just a glimmer in people's eyes, and the concept of install profiles was nowhere to be seen. Bryght worked on Drupal core to include multisite capability out of the box: all of a sudden, it was a bug if a module didn't support operating in a multisite environment.

Drupal-as-framework was in a much different state back then. Developers constantly hit cases where the core code didn't have enough "hooks and interfaces" to cleanly override everything. We ended up building a series of tools and daemons around Drupal in order to enable mass hosting. This tool is called HostMaster, and is Bryght's answer to mass hosting Drupal. It's built around Python and PostgreSQL, and has had the concept of "install profiles" for about 2 years. We originally had dreams of perhaps licensing or otherwise making money directly off this code. But in reality, this concept is foreign to our open source beliefs: the bits don't matter. Eventually, we put HostMaster under the GPL and made it available at https://svn.bryght.com/hostmaster (yes, you still need to request an account).

Pro Drupal Development: New Apress book on advanced development with Drupal

Thu, 30 Nov 2006 10:41:13 -0800

Woo-hoo! This is the book we've all been waiting for. Not to say that earlier books like Robert Douglass' Building Online Communities or Packt Publishing's Drupal books aren't good, but they cover a small slice of what Drupal has to offer, as well as targeting beginner to intermediate users with a range of needs.

If this book is going to be all that I think it is, it will prove an invaluable handbook on how to program with Drupal. It's clear that Drupal is becoming a framework, one that definitely needs to be on the selection list alongside things like Ruby on Rails.

If you're already a PHP programmer and you're about ready to select a framework you want to develop future web applications with, this will be the book for you.

Congrats to Matt Westgate and John VanDyk for birthing this monster. I'm really looking forward to it. Heck, I might actually learn how to program again (I trying to pick an IDE to use for Drupal/PHP development).

Presenting at BCTIA Go 2 Market Roundtable: Using the tools of Web 2.0

Wed, 27 Sep 2006 14:41:56 -0700

It's late notice, but myself and Andre Charland will be presenting on Using the Tools of Web 2.0 at the BCTIA Go 2 Market Roundtable tomorrow morning (Sept. 28th). It's taking place at the BC TIA Boardroom.

What are the tools of web 2.0? Well, the audience will mainly be marketing focused, so we'll cover some interesting marketing ideas and basic concepts around blogs, wikis, and customer interaction. That's code for "I need to work on some more slides tonight". I'll definitely cover how for small businesses, blog == Better Listings On Google.

[Cross posted from B. Mann Consulting]

IBM developerworks feedback, best practices for maintaining Drupal installs

Thu, 14 Sep 2006 23:06:45 -0700

I finally got around to stopping by the IBM developerworks forums and leaving some feedback for the folks that are doing the great series of articles on developing a site with Drupal. They're diving all the way down into developing modules, including explaining Drupal's hook system, which is great to see.