Tag Archives: API

Our MailChimp API Headache

It has been a while since I last posted on this blog. The main reason is that we have been so very busy for working and struggling with version 3 of MailChimp’s API.

Around two years ago we became aware of MailChimp’s latest API. It was certainly a very good example of a clean REST based API having all the characteristics of a well designed interface. All of the structures made sense and the anybody used to working with REST based APIs would have no difficulty in picking up this new API and creating a new powerful application with it.

So what went wrong?

About 18 months ago MailChimp informed the community of developers that it would be retiring all their other APIs in a year’s time so that they could concentrate on v3 as being the sole API. This meant shutting down not only V2 and older instances but also the export API.

This was a really big deal for us as Chimpegration made solid use of V2 and definitely the export API.  When we first gave MailChimp a demo of Chimpegration they told us that it was one of the most complex and intricate applications making use of their API. They were blown away by the detail available in synchronizing and the level of user choice that made up the application. What is more many of the features that exist today were not available at that time including the synchronization of groups, use of profiles and the ability to remove subscribers based on a query.

The news that MailChimp were going to shut down their API came as a shock to us. It meant some very big changes in Chimpegration without much to show for it. How can you possibly sell the fact that Chimpegration now uses V3 instead of V2. Nobody cares as long as the application carries on working.

What was clear to us when we started working with V3 was that those who designed this new API really had not spoken to anybody else about it. It was as if they had been working in a isolated area of the building, kept away from any V2ers or for that matter, existing users of V2.

They changed the structure of areas of the API such as groups. (The ids of V2 groups no longer match the ids of V3 making it very difficult to transfer to the new version unless we only consider the names of group items rather than the ids). They added a batch method so that you could send very many calls to MailChimp but they had no idea what the maximum allowable would be. We would wait, send through our batch updates and wonder if it would time out or not. They changed the unique identifier for a subscriber so that it had no bearing on the previous unique identifier. This meant that clicking on a link to bring up the contact in MailChimp no longer worked because the unique identifier no longer existed. (This has now been fixed, or at least we are redirected to the correct contact).

One of our biggest headaches which is still ongoing is around segments. The new API made it very difficult for us to look up a subscriber by constituent id (The Raiser’s Edge unique identifier). This, we had always insisted, be stored as a merge variable. Of course if we were starting from scratch we would use the new MailChimp unique identifier and store that in Raiser’s Edge but this would be an enormous change and one that would not work with existing organizations.

Previously we would feed in a segment into the method that would get the subscriber details. This segment would retrieve the subscriber by constituent id. This was removed and with it an enormous stumbling block was put in front of us.

Some relief was forthcoming about a month before the cutoff deadline. MailChimp announced that they would not be shutting down the export API. Clearly it was realised that the batch methods were just not good enough for large data transfers. This was important news for us as we relied heavily on this part of the older API.

It still leaves us with the issue of segments. The V3 developers did not look at the V2 segments. Instead they just changed the format. The problem, or course, is that the export API takes in segments in the V2 format. It is all very well fetching the user’s existing segments and then passing them into the export API (an extremely powerful feature that we have been making regular use out of) but all of a sudden we have to adjust each segment in order to convert it to a version that the export API understands.

What we would like MailChimp to do:

  • Either expose the V2 segment definitions again or allow us to pass in V3 segments into export. Either way should not mean that the existing method is just turned off. We need to time to convert and test.
  • Update the export API so that it is fully compliant with V3 (again keeping the existing calls backward compatible)
  • Produce a guide to transitioning from v2 to v3. I have been asking for this since we started and amazed that this was never forthcoming.

 

To our Raiser’s Edge Chimpegration customers: thank you for your patience. We realise that you do not care who is to blame for issues with Chimpegration as long as the application works. We understand that and, as ever, strive to release fixes to issues as soon as we are told about them. After much work we hope that we have reached a point where this latest version of Chimpegration is now more stable than ever. We continue to work on the plug-in to ensure it remains as useful as possible in integrating these two great applications.

Adding an Education Record to a Non-Constituent Individual

I have been struggling to find a way to a an education record to a non-constituent individual relationship. There is a Blackbaud knowledgebase article here that outlines how you add an education relationship but it takes a lot for granted. Here is the gist of it:

Dim oEdu As CEducation2
Set oEdu = New CEducation2
oEdu.Init REApplication.SessionContext
With oEdu
   .Fields(EDUCATION2_fld_RECORD_ID) = 678   'Selects the record to add the Education record to
   .Fields(EDUCATION2_fld_SCHOOL_ID) = "Berry College"
   .Save
End With

However where does the 678 come from? For a constituent this is simple. It would be the constituent system id found under the Records_fld_Id field.

However what would it be for a non-constituent individual relationship. After all they can also have education relationship records.

I tried the field Individual2_fld_Id but the error I got back told me that I could not add an education to an organization record! Not very helpful.

In the end, with a bit of trial and error I worked out that you need to put the field Individual2_fld_Relation_id.

This is the id for the corresponding entry in the Records table which consists of both constituents and non-constituents.

Hope this helps somebody.

Working with Phones/Email without addresses in Raiser’s Edge 7.94

Prior to the release of The Raiser’s Edge version 7.94 I was working with all of our products to ensure compatibility with this latest version. If you have not heard (and if you have not heard where have you been hiding), this release of RE removes phones and emails from physical addresses.

When the concept of emails was new, it was possible that you would have an email address tied to your telephone provider or an email address specific to your place of work.  Your phone would either be at home or at work. Having these connnected to your physical address made sense. However it quickly became aparent that with the arrival of mobile phones and of email addresses that were accessible no matter where you were located, phones and emails (and for that matter all types of communication links) should be tied directly to the constituent record and not to a physical address. This is what has happened with the release of RE7.94.

This is a big shift and in terms of developing applications, we have had to allow for both possibilities so that our programs are compatible with users still on 7.93 and below and those that have made the leap over to 7.94.

The good news is that the old way of doing things still works in 7.94. You can still access phones via the CConstitAddress.Phones collection for an address. However you will probably want to access them how they are intended… Free from addresses.

This is done using the new interface IBBPhonesParent. This is implemented by CRecord, CIndividual2 and COrganization2. The collection of phones is a CConstitPhones object which contains the usual methods. You can iterate the collection to give you one CConstitPhone object but here is the problem.

For reasons that I don’t fully understand (I was told due to binary compatibility reasons) there are no properties or methods on the CConstitPhone object. Instead you have to convert this object to an IBBDataObject in order to access the Fields property. This is a real pain but to save myself some trouble I put together two extension methods for the CConstitPhone object which does this for me. (Unfortunately extension properties do not exist so that is why I cannot simply create an exact corresponding Fields properties. Those working with C# will be familiar with this as there is not a Fields property but rather  get_Fields and set_Fields methods)

<System.Runtime.CompilerServices.Extension()> _
Public Function Fields(ByVal phone As CConstitPhone, fieldConstant As ECONSTITPHONEFIELDS) As Object
   Dim dataobject As IBBDataObject = CType(phone, IBBDataObject)
   Return dataobject.Fields(fieldConstant)
End Function
<System.Runtime.CompilerServices.Extension()> _
Public Sub Fields(ByVal phone As CConstitPhone, fieldConstant As ECONSTITPHONEFIELDS, value As Object)
    Dim dataobject As IBBDataObject = CType(phone, IBBDataObject)
    dataobject.Fields(fieldConstant) = value
End Sub

With these extension method you can simply access the Fields methods on a CConstitPhone object in almost the same way as you would with other objects.

So here is an example of creating a new phone on a constituent record using the above extension code.

Dim constit As CRecord = GetConstituent()
Dim phonesParent As IBBPhonesParent
Dim phones As CConstitPhones
Dim phone As CConstitPhone

phonesParent = CType(constit, IBBPhonesParent)
phones = phonesParent.Phones
phone = phones.Add()
phone.Fields(ECONSTITPHONEFIELDS.CONSTIT_PHONES_fld_PHONETYPE, "Home")
phone.Fields(ECONSTITPHONEFIELDS.CONSTIT_PHONES_fld_NUM, "123-4567")
phone.Fields(ECONSTITPHONEFIELDS.CONSTIT_PHONES_fld_IS_PRIMARY, True)

constit.Save()
constit.CloseDown()
constit = Nothing

Overcoming the password update policy for custom Raiser’s Edge applications

A recent question on the Blackbaud forums got me thinking about this problem. The issue is this. If an organisation has a password policy in RE in place that ensures that users have to update their password every X days, what happens to a custom application that runs every day in order to perform some maintenance / export / import etc.? It is also required to update its password. This is somewhat problematic because most scheduled tasks are just meant to be run and more or less forgotten about.

The obvious solution is to turn off this functionality. However you are only able to do this for the whole organisation which is problematic.  Another solution is to use Windows authentication to log into RE. That way it is Windows that decides the password policy. This is also not always possible.

Here is a third, programmatic way of doing this. You need to make use of the “other” API. I have mentioned this previously in Checking Security. You need to make use of the  Blackbaud.PIA.RE7.SecData7 assembly. This gives you access to the security objects that are not present in BBREAPI.

You set up a database table with three columns; a primary key id, a password and an expiration date. You then fill the password column with a list of passwords that could be used.

On starting the application you select from the table the password with the most recent expiration date (which may be in the future). You use this to log into RE using the usual code. Once that is done you determine whether or not you need to change the password. If the expiration date is in the past then you should change the password using the code below. The new password should be the next password in the table that has a blank expiration date and lowest id.

Dim user As New CUser
user.Init(SessionContext)
user.Load(SessionContext.CurrentUserID)
user.Fields(Blackbaud.PIA.RE7.BBInterfaces.EUSERFields.USER_fld_PASSWORD)

You then set a new expiration date on this password. This expiration date should be a good few days before the actual date you are required to change the password that way the existing password will still be good.

If there are no passwords left in the table you can remove all the expiration dates and start from the first value in the table i.e the password with the lowest id. I assume that RE allows you to use the same password as at some point in the past if not the most recent values.

One variation to this is to just create a random passwords and not have a list of passwords. That way you would only have a table with one row and an expiration date.

Any improvements? let me know in the comments.

Checking access restrictions in The Raiser’s Edge

In a recent project I had to ensure that specific confidential information was being saved as an action. I had a custom screen where this data was going to be viewed and edited. Only certain users had access to that action using security by action types. I had to check to see if the current user was able to view the action and if they could whether or not they were able to edit the details. There are useful methods for this in the API and I assumed it would be a simple task to use them. Continue reading Checking access restrictions in The Raiser’s Edge

Batch API – error with tempRecords is nothing

“Nothing :
Init method must be called before using this object.” My colleague was testing a new application that I have written for a client. Every so often she would tell me that this error message was appearing in the control report. Everything else appeared to work. Just what did this mean? It was one of those classic RE error messages that means nothing. When I started to debug the problem I saw the error quite soon. Again it was one of those Batch API idiosyncrasies that just needed to be dealt with.

I was creating several gifts in a batch. I was looping through the gifts and creating each one.

For Each donation In donations
    tempRecords = CType(batch.TempRecords, Blackbaud.PIA.RE7.BatchData.CTempRecords)
    PopulateOneGift(constitSysId, batch, tempRecords.Add, donation)
    tempRecords.Save()
 Next

The error occurred on the second iteration of the loop. Whenever the tempRecords.Add
was called the error was raised. This was strange because normally you get this kind of error when you have not initialized the object or done a Closedown on it. Anyway the solution to add one line before calling the Add:

For Each donation In donations
    tempRecords = CType(batch.TempRecords, Blackbaud.PIA.RE7.BatchData.CTempRecords)
    tempRecords.Reload()
    PopulateOneGift(constitSysId, batch, tempRecords.Add, donation)
    tempRecords.Save()
 Next

That seemed to fix the problem.

EDIT: or so I thought. It appears as though it worked once then failed after that. I have now resorted to saving and closing the batch after each iteration and then opening it up again at the beginning. Ah the joys of the batch API.

The Raiser’s Edge Integrated with Potentiality… A series of case studies (2)

Towards the end of 2010 we started working with Potentiality to integrate their system with The Raiser’s Edge. At the time I knew very little about them and their products. Their headquarters is in Melbourne, Australia with an office in London. Even though they have an office in London our main contact was with their Melbourne office. Their product was a community based content management system (CMS) targeted towards schools and universities but also used by any organisation that was trying to build up a community of supporters.

Continue reading The Raiser’s Edge Integrated with Potentiality… A series of case studies (2)

The Raiser’s Edge Integrated with Gmail… A series of case studies (1)

This is the first in a series of case studies looking at custom integration of Blackbaud products.

Unlike very many of our previous third party integration application the main driving force behind this application, Biographica, was the fact that we could do it. We did not have any particular client knocking on our door asking us link the two products and Google , funnily enough, did not approach us either asking us to develop an integration between The Raiser’s Edge and Gmail.

Continue reading The Raiser’s Edge Integrated with Gmail… A series of case studies (1)

Documenting VB.NET and C# Code

If you are using VB6 then you are possibly somewhat limited as to the way you can document your code and then have it published automatically. (I say that from complete ignorance and somebody is very welcome to show me otherwise). However in VB.NET and C# there is the built in XML documentation. In both languages you simply place the cursor the line above the field, property, method or class that you wish to document. In VB.NET you type: Continue reading Documenting VB.NET and C# Code