Tag Archives: Chimpegration

GDPR and Consent in Raiser’s Edge

I have been really busy of late. While many an EU non-profit have been kept awake at night because of GDPR, I have not quite been kept awake but nevertheless been very involved in implementing GDPR into our products. This has taken the form of the new consent module.

If you are not in the EU or otherwise not on the latest version of RE7 then you may wonder what I am talking about when I mention the new consent module. I am not going to go into too much detail here as Blackbaud have some good resources that cover it here.

What I will say is that the implementation of consent is very different from many other modules in RE. Luckily there is less and less scope for RE7 API developers as Blackbaud moves towards NXT and expands the REST based SKY API. So I am wondering if as, a last challenge towards those remaining in the RE7 API game (myself included), Blackbaud decided to make the new consent module even less consistent than previous modules.

Here are a few of its features:

  • The consent collection cannot be found within the regular BBREAPI assembly. You have to look elsewhere for it.
  • It does not save alongside the rest of the constituent records but has its own save routines. What this also means is that the VBA events are not fired when a consent record is saved.
  • In the first release it does not cause an exception when you do not supply a valid combination of channel and category as it does in the UI but if you are really clever (or decipher the sample code), you can determine how to do your own validation.

Now I should not be too harsh on the BB developers. Introducing a new module like this is extremely difficult. There are so many intertwined areas that must be accounted for and I am sure that the design decisions were taken for a reason. (One of which being that it is much easier this way to work with very many consent records if they are a standalone entity)

How are we updating our applications to work with the consent module?

Audit Trail:

As you would expect changes made on consent records will be tracked but because consent records are saved as a standalone entity they will only be saved if the constituent record is also saved afterwards.

Validatrix:

This is a tricky one. We have included consent records as part of Validatrix but because they are a standalone entity and are open and saved in their own rights, they do not fire the VBA events that tell Validatrix to prevent a save. That means that a user can add a consent record and shut the constituent without saving the constituent. You cannot therefore have a consent record as the primary criteria. However, if you have the consent record as a dependency of a constituent based field then it will be included in the criteria when you save the constituent.

Importacular:

As you would expect, Importacular allows you to import consent records. You can match on any combination of channel, category, date, response and source to ensure that you are not creating duplicate consent records (although by default it matches on channel, category, date and response).

Chimpegration:

This is perhaps our most ambitious development. Until Blackbaud add consent information to query and export you are not able to export consent records to MailChimp. However it is probably more useful to export the outcome of the consent records i.e. solicit codes which show you a good picture of a constituent’s intentions.

On managing campaigns you can add a consent record based on the action i.e. if a subscriber unsubscribes you may want to add a consent record.

Sync is where the most complex piece of development occurs. We allow you to map individual groups and group items to the addition of different types of consent records. Equally when specific solicit codes are added (in response to consent records being added previously) these can be mapped to group items. We have a longer description of this on our knowledgebase.

When is this available? Importacular, Audit Trail and Validatrix are already live. Chimpegration is live for self-hosted and will go live in the near future for hosted organisations.

Moving over to SKY API

This blog has mainly consisted of the COM based RE7 API and my musings of all things Blackbaud related. I am sure that the latter will no doubt continue but I have realised for a while now that as time goes on the RE7 API is becoming less and less relevant (although not entirely so) and that there is a natural progression towards the SKY API.

In general there is probably less to be said about SKY API. Firstly Blackbaud have been doing a much better job at documenting it than they ever did with the RE7 API. They are also putting a lot more thought into it so that there are far fewer inconsistencies (so far at least) than there ever were with the RE7 API. Where there are difficult, new, areas they have written up good documentation or blog posts to explain. In short they have made my job here somewhat redundant… Well thanks a lot Blackbaud!

Actually, yes, thank you. I would much rather a clean usable API than one where I have to write up blog posts explaining how to do things. I am sure that there will be moments but there will quite possibly be fewer of them.

One topic that I think deserves some discussion though is the porting of existing functionality from RE7 to SKY. Those of us that have products written for RE7 are keen to see the functionality available on SKY in order that we can port the solutions over.

I have been very keen to transfer Chimpegration but one of the stumbling blocks has been the lack of bulk data processing on SKY. Specifically the ability to return filtered lists of constituents. On many platforms this means simply specifying a last data changed or a keyword search. On RE7 though it is possible to make use of a query to retrieve that information. The user would themselves set up the criteria in the query and the application would allow the user to select that query.

There is some discussion on the SKY API forums about how imperative it is that this be ported over to SKY and that we should be allowed to once again select an existing query.

Despite really needing this functionality for Chimpegration, I am not convinced that this is the best course of action for SKY. This new API should embrace a general approach to this problem. It cannot be based on RE7’s query module. There is definitely a need to generate lists based on complex filters and criteria and that should be exposed somehow to the developer community but to simply port the existing RE7 functionality to SKY would be short-sighted and not take into consideration the other applications what will one day make use of the same API. I want an API that will work with RE, BBCRM, ETap and others. To simply port queries to SKY would confuse the issue and make for an API that is not consistent.

Our MailChimp API Headache

It has been a while since I last posted on this blog. The main reason is that we have been so very busy for working and struggling with version 3 of MailChimp’s API.

Around two years ago we became aware of MailChimp’s latest API. It was certainly a very good example of a clean REST based API having all the characteristics of a well designed interface. All of the structures made sense and the anybody used to working with REST based APIs would have no difficulty in picking up this new API and creating a new powerful application with it.

So what went wrong?

About 18 months ago MailChimp informed the community of developers that it would be retiring all their other APIs in a year’s time so that they could concentrate on v3 as being the sole API. This meant shutting down not only V2 and older instances but also the export API.

This was a really big deal for us as Chimpegration made solid use of V2 and definitely the export API.  When we first gave MailChimp a demo of Chimpegration they told us that it was one of the most complex and intricate applications making use of their API. They were blown away by the detail available in synchronizing and the level of user choice that made up the application. What is more many of the features that exist today were not available at that time including the synchronization of groups, use of profiles and the ability to remove subscribers based on a query.

The news that MailChimp were going to shut down their API came as a shock to us. It meant some very big changes in Chimpegration without much to show for it. How can you possibly sell the fact that Chimpegration now uses V3 instead of V2. Nobody cares as long as the application carries on working.

What was clear to us when we started working with V3 was that those who designed this new API really had not spoken to anybody else about it. It was as if they had been working in a isolated area of the building, kept away from any V2ers or for that matter, existing users of V2.

They changed the structure of areas of the API such as groups. (The ids of V2 groups no longer match the ids of V3 making it very difficult to transfer to the new version unless we only consider the names of group items rather than the ids). They added a batch method so that you could send very many calls to MailChimp but they had no idea what the maximum allowable would be. We would wait, send through our batch updates and wonder if it would time out or not. They changed the unique identifier for a subscriber so that it had no bearing on the previous unique identifier. This meant that clicking on a link to bring up the contact in MailChimp no longer worked because the unique identifier no longer existed. (This has now been fixed, or at least we are redirected to the correct contact).

One of our biggest headaches which is still ongoing is around segments. The new API made it very difficult for us to look up a subscriber by constituent id (The Raiser’s Edge unique identifier). This, we had always insisted, be stored as a merge variable. Of course if we were starting from scratch we would use the new MailChimp unique identifier and store that in Raiser’s Edge but this would be an enormous change and one that would not work with existing organizations.

Previously we would feed in a segment into the method that would get the subscriber details. This segment would retrieve the subscriber by constituent id. This was removed and with it an enormous stumbling block was put in front of us.

Some relief was forthcoming about a month before the cutoff deadline. MailChimp announced that they would not be shutting down the export API. Clearly it was realised that the batch methods were just not good enough for large data transfers. This was important news for us as we relied heavily on this part of the older API.

It still leaves us with the issue of segments. The V3 developers did not look at the V2 segments. Instead they just changed the format. The problem, or course, is that the export API takes in segments in the V2 format. It is all very well fetching the user’s existing segments and then passing them into the export API (an extremely powerful feature that we have been making regular use out of) but all of a sudden we have to adjust each segment in order to convert it to a version that the export API understands.

What we would like MailChimp to do:

  • Either expose the V2 segment definitions again or allow us to pass in V3 segments into export. Either way should not mean that the existing method is just turned off. We need to time to convert and test.
  • Update the export API so that it is fully compliant with V3 (again keeping the existing calls backward compatible)
  • Produce a guide to transitioning from v2 to v3. I have been asking for this since we started and amazed that this was never forthcoming.

 

To our Raiser’s Edge Chimpegration customers: thank you for your patience. We realise that you do not care who is to blame for issues with Chimpegration as long as the application works. We understand that and, as ever, strive to release fixes to issues as soon as we are told about them. After much work we hope that we have reached a point where this latest version of Chimpegration is now more stable than ever. We continue to work on the plug-in to ensure it remains as useful as possible in integrating these two great applications.

What is the difference between Chimpegration and Online Express?

What is the difference between Chimpegration and Online Express?

We are big Blackbaud fans and love much of what Blackbaud produces. Online Express is no exception. The fact that there is one application that can ask for donations, handle event registrations, manage memberships as well as allowing you to send out targeted and personalised emails is amazing. What is even better is that it does so with a complete integration into The Raiser’s Edge. – WOW!

Continue reading What is the difference between Chimpegration and Online Express?

Chimpegration on Blackbaud Hosted Raiser’s Edge (and NXT)

I cannot think how many times I have had to write back to a prospective client with the stock phrase “Unfortunately our products are not available on the Blackbaud hosted environment at the current time”.  The ground-breaking news is that after at least 7 years waiting in the wings, we are able to release one of our products on the Blackbaud hosted environment. (Finally I can updated my stock phrase)

When it was first released Chimpegration revolutionised the meaning of integration within The Raiser’s Edge world. It was the first app to fully embrace the potential of two separate platforms to transfer data between Raiser’s Edge and MailChimp. Unlike some integration applications it takes data directly from the source. Data flows from MailChimp via Chimpegration into The Raiser’ s Edge and back again. There is no need for any manual manipulation along the way. Chimpegration gives you a multitude of options to help smooth the different data points and ease the transition of information from MailChimp, an email marketing suite, to The Raiser’s Edge and donor management CRM system.

Azadi Sheridan, independent fundraising data expert, RE guru and former Blackbaud senior manager said,

‘Integration’ can mean different things for different products, but Chimpegration is the most complete and automated integration for MailChimp that I’ve seen for any fundraising database.

And now all of this is available to Blackbaud hosted clients too!

What does this mean for the future? One of the issues that many of the prospective clients faced and even our existing clients was whether they should invest in technology that is not available to them on NXT. They would put off a purchase or renewal decision because they wanted to wait and see what NXT would hold. RE7 has so much functionality but NXT looks amazing.  Having the best of both worlds is certainly a selling point for any organisation. With Chimpegration on the NXT hosted platform it is easier than ever to transition across to NXT and keep working with MailChimp.

So now you have heard all about it why not request a demo of Chimpegration.

Ready to take the plunge? Request a trial of Chimpegration on hosted.

If you are not hosted and have questions please get in contact too. Those who are not hosted by Blackbaud can also download and trial the Pro version of Chimpegration now from our Chimpegration page.

Chimpegration shortlisted for a Digital Fundraising Award – Vote for us

Chimpegration, our plug-in that integrates Blackbaud’s The Raiser’s Edge and MailChimp has been shortlisted for a Digital Fundraising Award. We are really excited about this. So many organisations are using both The Raiser’s Edge and MailChimp and can now integrate the two for free with Chimpegration.

Vote now and tell all your friends and colleagues to vote too for Chimpegration!

http://www.digitalfundraisingawards.co.uk/digitaltool

MailChimp Offers Non-Profit guides – Works well with Chimpegration

At Zeidman Development we are big fans of MailChimp. We use it ourselves to send out our newsletters and we have integrated it with The Raiser’s Edge in the form of Chimpegration. MailChimp have now hired somebody to help non-profits make better use of MailChimp. Their latest blog post explains how to use merge variables in MailChimp.

Users of The Raiser’s Edge should be used to using merge variables given the built in tools that link RE to Word. The article explains how you use the corresponding functionality in MailChimp. What is really good is that if you set up merge variables in MailChimp you can easily link those variable to query output in Chimpegration. In the free version you are able to export your data to MailChimp and in the Pro version you can synchronize the data sets. That means you can use MailChimp merge variables as an online data source. If you tie it into your website, users can log in, update their merge variables themselves and all that is fed back into The Raiser’s Edge on synchronization. Sophisticated, flexible integration in one product.

For more information about Chimpegration take a look at our site: http://www.zeidman.info/downloads/chimpegration.php

Greater Segmentation with MailChimp and Chimpegration

Chimpegration Professional allows users to see a constituent’s email activity.  It shows which emails they received, what they opened and which links they clicked on. Chimpegration, both free and Professional allows you to segment a list on MailChimp‘s by exporting constituents into specific groups. However there is no way currently in Chimpegration to segment based on this activity. Chimpegration gives you a view of the MailChimp activity. It does not bring the data over into The Raiser’s Edge so it is not possible to segment on this data.

However, now MailChimp have released a new product Goooal. You can use Goooal in combination with your existing MailChimp lists to help segment your constituents. Whenever a person clicks on a link you can setup Gooal to put that constituent into a specific segment. Once they are in a segment you can target them based on their previous visits. If your newsletter has links to information on different areas of your cause this is a perfect way to target those constituents who are interested.

Goooal is a free addition to MailChimp and compliments Chimpegration’s seamless integration with The Raiser’s Edge.

The challenges of developing generic solutions for The Raiser’s Edge

We develop a number of different types of solutions that connect The Raiser’s Edge with third party applications. When this is done for one organisation the hardest challenge is getting the requirements to match the end user’s as closely as possible. However we also develop a lot for third party applications directly. Most recently we released Chimpegration but there have been many others that integrate many different areas of the application. With these clients the important point is not so much as to match one set of requirements exactly but to match as many requirements exactly!

I have yet to come across two organisations that have set up RE the same way. Sometimes the differences are small but sometimes they are very large. One recurring theme is that of phone/email types. The Raiser’s Edge seems to be relatively unique in its setup of these values. Not only are phones and emails stored in the same location but you can store them according to address type too. How does this match up with a third party web application that uses email address as a primary key? They may have one field for home phone, one for mobile and possibly one other. How do you get that tie in with the possibility of any combination of phone and email types? I have seen a whole plethora of regular phone types e.g.

Home
Business
Preferred
Home 1
Home 2
(etc)
Primary
Work
Company

Those are just the ones I can think of as I write this. And of course each of these could have a number after them.

Then of course there is the proliferation of email (or is it e-mail, etc) addresses.

What techniques do we use to overcome these issues? When we are working with a third party developer directly it is often in their interest to develop the configuration piece. This saves them a lot of development cost. This means that we simply say to them if you want us to update a phone number you need to tell us which type it is. This is then supplied in the file/webservice. Likewise when we supply that piece of data, we also send the phone type too.

Another solution is to build a configuration part to the application. This is what we did with Chimpegration. We allow the end user to map the fields that they want to synchronise so that they specify which fields on MailChimp map to the fields on The Raiser’s Edge.

The last solution is the least desirable. It is possible to simply say that the home phone number should be called “Home” and email address should be called “Email” – end of story. This is clearly the simplest and cheapest but unless you have a lot of sway over the organisations that you are selling to it is unlikely that you are going to get many buyers.

Phone and email types are perhaps the most obvious but what other issues can arise?

Where you are collecting business details, should these be added to an organisation or to a business address on the constituent record? Should you create a new constituent for the organisation? How can you be sure that it does not already exist in the system but under a slightly different name. One solution is to allow the end user to review the matches that have been made but this again adds to the development cost. It could also be prohibitive if the volume of data you are bringing in is large.

What about fields that do not have an obvious place in The Raiser’s Edge. One application that we worked with had an anniversary date. There is no such field in RE. We gave the end user the option to ignore the field, store it has the spouse relationship from date or store it as an attribute.

What happens if one organisation makes a field mandatory? When I work with an organisation directly I will ask them what mandatory fields they have set up on their system. If, as part of the process, I have to create a new constituent then I will ask them to give me a default value for that field. For a generic solution this has to be worked into the application configuration.

You can see much of this in action in Chimpegration where we account for mandatory fields and different combinations of phone types and emails. Check out the synchronisation screencast for a glimpse of this in action.

So all said and done is this type of solution to be avoided? Absolutely not! It is not cheap because of the extra work involved in making the application work for all types of organisations. When people complain about NetCommunity or Patron Edge integrating badly with The Raiser’s Edge hopefully this article will have given you some insight as to the skills required by Blackbaud in getting the integration to work well. (Update 15th Dec 2011: I should clarify that given the difficulty in developing generic solutions I actually believe that Blackbaud have done a good job with these integrations)

If you are a third party application wondering how to integrate with The Raiser’s Edge then speak so us. We are skilled at doing this and have done it a lot. We can either do it for your or we can share the development. When done well it is a great asset to your company and will bring Raiser’s Edge users to your application.