View source for DevSummit07:Integration Experiments: What's Happening, What's Working, What's Next?

=== Facilitated by [http://dotorganize.net/ DotOrganize] and [http://onenw.org ONE/Northwest] ===

The world of tools integration has moved from theory to practice. Clients are mashing our tools together, vendors are experimenting with APIs and deeper integrations, and consultants are building whole practices around integration. Have you integrated your tool with any others? What lessons did you learn? Will you do more? What are your clients asking for?

Come share real world case studies of integrations, and help draw out the common threads and lessons that can make future integrations easier and more effective.

Be prepared to summarize your experience in under three minutes and to learn something new; we want this to be a result-driven conversation, not just another "integration is great!" blab-fest.


===Session Notes===
What are we actually doing in terms of integration?  It’s easy to be stuck in the ether, but let’s take it down a level – let’s understand what we’re doing, and tease out the common threads, of what’s working.

'''Chris, Democracy in Action''' Grizzled veteran of many integrations

* Originally, API wasn’t stable
* Partners that they were integrating with were trying to hit as if they would something like Salesforce
* But they kept adding things, which made the API unstable – every time they added stuff, broke the API
* Lesson learned: if you’re going to publish it, have a stable release schedule
* Version the APIs, with a staging API
* Be wary of things that have frequently updated APIs 
* Needs to be a slow release schedule, so that people have time to develop and then have it live for awhile before you need to change it.  Should be guaranteed to work for some period of time, maybe six months
* Implement deprecation periods – nothing can be removed, can’t break things.  Need a six month warning – get a warning in their log, ideally, that things are going to change.  Won’t preclude people being surprised, but at least you can say you warned them.
* Ideally would have had an API manager, managing communications with partners, tracking who was using 
* How long did it take to build the API?  Everything they built was the API – so whenever they changed the system, the API changed.
* API isn’t something that you build once, not a feature, but is an ongoing communication thing, needs to be supported ongoing
* People wanted to integrate using the API, asked for access, and it seemed like no big deal to give it out.  But when they updated, API broke, and people freaked out.


'''Steve Andersen and Jon Stahl, ONE/NW'''  workflow of interest, work can’t be done in one system

Plone to Salesforce integration
* Wanted to be able to track website members/ users in the Salesforce database 
* There is a Salesforce SOAP webservices API which is core to their business, very strong
* Plone was structured to be able to do LDAP integration, other plug-ins for people
* So built a plug-in so that when someone registers, the person is automatically added to the Salesforce database
* Harder to hit webservices than just a database sitting on your server – latency, downtime (need to think about caching and adding later), thinking about duplicates (maybe they’re already in but don’t know it)
* Process lesson learned:  was critical to be very thoughtful of the user stories that they’re trying to support – new user, login, updating your salesforce profile in plone
* Important to limit your release to something small and manageable
* Things took more time to investigate than they expected – for instance, to understand the Salesforce API and all the options it offered
* $30,000 time and money, both internal (1/3) and consultants (2/3).  Started summer, had a .9 release in October.  Not yet released.
* Chose to do transactional based rather than full integration tables because of effort – a lot less effort 
* Probably wouldn’t have taken a lot less time to do specific to one client rather than generalized.  Generalized was the right was to do it, regardless.

'''Ryan Ozmiek, PicNET.'''  Nonprofit Soapbox.  Trying to integrate with as many things as possible.  

Nonprofit Soapbox connections
* Salesforce – add people from CMS to SF, update profile in CMS but stored in SF, directory of people shown in CMS.  
* Authentication and roles left on the CMS side.  There might be a reason to integrate that – for instance, if you wanted passwords managed jointly – but it’s complex, and neither Ryan nor the ONE/NW 
* CiviCRM – their code is sitting within Joomla database
* DIA 
* Odeo
* Constant Contact
* All of these integrations were done as individual apps, but using a similar architecture.  Created a middleware for each application that allowed people to hit it as if it was the Joomla database
* The integration is getting easier, so for Salesforce – 3 weeks, 1 developer 
* Lessons learned:  really useful to have precedents, for someone to have done it before with  a similar system.  Also to have a key contact for the application

'''Andy''', La Leche project

* Webservices uses a different type of data concept than a SQL database.  Webservices – remote data invocation – tightly binds the purpose for which you are requesting data to the data itself.  This makes it good for application integration, but less good for data integration.
* La Leche project – 10 systems with duplicate data, all messed up, and in addition wanted to bring in additional systems
* Kintera Sphere, abstracted identity layer (single sign-on) shared between, a number of additional data sources
* Used the XDI stack


'''Rob Miller''', Plone with LDAP, Plone with Filemaker

* Key to interoperability is lots of small pieces that do one thing very well, and might behave different depending on its environment.  If one thing is doing a lot of things, it will cause blockage.
* There’s a lot of things inside of Plone – wikis, mailing lists
* Want to integrate with WordPress, ticket tracking application
* Origiinally tried to use Plone as the central entity, but found that that caused blockage, so did a looser coupling.
* Created a deliverance layer that takes unformatted data and meshes it together into one visual feel.  Big advantage – easy to change look and feel, and don’t need to know anything about the stack in order to make changes
* But the applications also talk to each other – so for instance if you’re trying to log into WordPress, goes back to Plone to get the cookie and then 
* Plone is the central owner for member/ authentication information, but otherwise the applications don’t communitcate with each other, but instead the deliverance layer does routing, business rules, and packages everything into a consistent front end.

'''Tate, with Dot Organize.'''  Dot Organize is trying to catalyze a solution to nonprofit integration woes.  VAN (Voter Action Network) and Catalyst

'''Greg, Metacalender.org'''  Integrating all the events from a region for progressive organization.  Going live soon in LA.

'''Nika, Skyline'''  Helping orgs to choose CMS and email systems.  Blue Utopia (integrated CRM and CMS)

'''Michel, CiviCRM'''  Many of his projects involve low level integration, some more successful than others

'''Jeremy'''Has done CMS/ CRM integrations, finding the right suite.  There’s no silver bullet.  Intranets and document management 

'''Marnie, TechSoup''' working on integrating TechSoup – not just technology, but a person issue - can workflow help?

'''Dave, Grassroots.org''' integrating Mailman, among other things

'''Evan, NPower Seattle.'''  Plone, Salesforce and Google Maps 

'''Laura, Idealware.'''  Interested in the different models for integration

'''Simon, Direct Leap Technologies.'''  GoogleMaps mashup with phone list, integrating with tools that you store phone lists in

'''Dan, Community Software Lab''', Has integrated Mailman

'''Leda, Dot Organize'''

'''Holly, NTEN.'''  Never done an integration, but critical to nonprofits

'''Angela, Grassroots.org'''

'''Jim, Chicago Tech Co-op''' working on a giant mashup of a number of different sources,