Test-Driven Development Example

The following post is based on an email I sent a student worker who, bless his heart, is doing terribly complicated and important, but also terribly boring, grunt work. As a thank-you, I'm giving him Apex coding projects to do in a Developer Org and granting him a block of time per day to work on them.

---

This morning, I'm working on writing a new "public" Apex method that, as a parameter, takes a set of Contact IDs. If a Contact's "Account.Name" is meaningless (e.g. "No Company Assigned,"), this method then checks to see if there is a non-null string in that Contact's "Company Holding Spot" custom field. If any such strings happen to match the Name of a record in our Accounts table, my new method populates the Contact's AccountId with the appropriate value. Hopefully, I can use this to clean up a bunch of old data in one fell swoop.

I'm writing the new method in test-driven-development style, which means I have my test fully ready to go before I've put any "action" code into my new method. Here's what that looks like:

@isTest
public class ScratchPadClassTest {
    
    // Class-level static variables
    private static Boolean setupAlreadyRan = FALSE;
    private static UtilityDefaultInfoOftenNeeded useful;
    
    // The test method    
    static testMethod void testCompanyHoldingSpotToAcctId() {
        runSetup();
        
        // Set up a couple of test accounts (also remember that a 3rd "No Company Assigned" one exists thanks to "runSetup()")
        Account aMcD = new Account(Name='McDonalds');
        Account aTgt = new Account(Name='Target');
        insert new List<Account>{aMcD, aTgt};
        
        // Set up a couple of test contacts who newly claim they work at McDonald's (but one of whom already worked somewhere else)
        Contact c1 = new Contact(LastName='McTestNullAcctNowMcD', AccountId=useful.getDefaultAccountId(), Company_Holding_Spot__c = 'mcdonalds');
        Contact c2 = new Contact(LastName='McTestTgtAcctNowMcD', AccountId=aTgt.Id, Company_Holding_Spot__c = 'Mc. Donalds');
        insert new List<Contact>{c1, c2};
        
        Test.startTest();
        // Call our data-transformation method on the IDs of our two contacts
        ScratchPadClass.copyCHSToAcct(new Set<Id>{c1.Id, c2.Id});
        Test.stopTest();
        
        // Pull a fresh copy of our contacts out of the database
        Map<Id, Contact> csAfter = new Map<Id, Contact>([SELECT Id, AccountId, Account.Name FROM Contact WHERE Id IN (:c1.Id, :c2.Id)]);
        
        // PERFORM TESTS CHECKING FOR DATA QUALITY
        // Since Contact #1 had a generic "No Company Assigned" account to start with, McD's should have propagated into AccountId
        System.assertEquals(aMcD.Id,csAfter.get(c1.Id).AccountId, 'c1 Account Name is ' + csAfter.get(c1.Id).Account.Name);
        // Contact #2 was already working at a real company (Target), don't overwrite w/ McD's.  Needs human review.
        System.assertEquals(aTgt.Id,csAfter.get(c2.Id).AccountId, 'c2 Account Name is ' + csAfter.get(c2.Id).Account.Name);
    }
    
    // A private helper method
    private static void runSetup() {
        if (setupAlreadyRan == FALSE) {
            DefaultTestDataFactory.setUpCustomSettings();
            if (useful==null) { useful = UtilityDefaultInfoOftenNeeded.getInstance(); }
            setupAlreadyRan = TRUE;
        }
        return;
    }
       
}

NOTE: You might notice that I have classes called "DefaultTestDataFactory" and "UtilityDefaultInfoOftenNeeded" whose code isn't shown here. Don't worry about them. All they do that matters is to this code is:

1. Insert a "No Company Assigned" account into the database (DefaultTestDataFactory) and
2. Provide an easy way to retrieve the ID of that account (UtilityDefaultInfoOftenNeeded.getInstance().getDefaultAccountId())

The body of ScratchPadClass looks like this:

public class ScratchPadClass {

    public static void copyCHSToAcct(Set<Id> cIds) {
        return;
    }
    
}

When I "Run Test" on "ScratchPadClassTest," the first System.AssertEquals fails with the following error message:

• Assertion Failed: c1 Account Name is No Company Assigned: Expected: 00738000006UTiPEDC, Actual: 00738000006UTiOEDC

In other words, I expected c1’s Account.Name to change from "No Company Assigned" to "McDonalds," but it didn’t happen.

Well, of course it didn’t happen. ScratchPadClass.copyCHSToAcct() doesn’t even do anything yet!

Now I will go write ScratchPadClass.copyCHSToAcct(). And I’ll know I wrote it correctly when my test passes.

And THAT’s test-driven development!

Define and Initialize a Map, List, and Set in Apex

I use Dave Helgerson's blog post "Define and Initialize a Map, List, and Set in Apex" pretty much every time I code. Keep it in your bookmarks and you can essentially type "map" or "list" or "set" in your browser URL and have a quick-reference whenever you need it.

Creating Salesforce custom objects and custom fields with code (the Metadata API)

Thanks to StackExchange, I got the help I needed when I last posted about creating custom objects and fields with code.

My department at a university bought a Salesforce org 5 years ago. That's the one I'm the sysadmin for. We meant it to fill a lot of gaps left for our department by the university's central ERP database (Banner) - such as Banner not handling week-long "courses" very well, Banner not handling "deans' corporate rolodexes" very well, Banner not playing well with modern mass-emailing/texting/etc. software, etc.

Our university's central IT department just bought a Salesforce org about 1 year ago. We're not sure yet exactly how widely it will be used, but for starters, it's replacing our home-coded web-based application system (which used to dump straight into our ERP).

Because of that limited scope, it really doesn't yet fill any of the "holes" in the central ERP's offerings that my department bought its own Salesforce org for. Which means we're leaving both of them live.

And now instead of just worrying about integrating a daily dump from the ERP into our Salesforce org, I also get to set up a daily dump from the new central-Salesforce into our Salesforce.

---

Our typical pattern for "daily dumps" is "dump the data where it can't hurt anything, and move it where it belongs by trigger/workflow/process afterwards."

So, for example, "First Name" from Banner dumps into Contact.Banner_First_Name__c - and then is post-processed to fill out Contact.FirstName if there isn't already something there. (List View reports help us find & correct cases where the two fields have different values.)

I was about to dump 20 never-before-dumped fields' worth of data into Contact and about 6 objects (at 4-20 fields apiece) from yet another database into our Salesforce, so I had a lot of "creating objects & fields" to do before I could even get started setting up the Jitterbit feed between them.

What really bugged me was the idea that because I was essentially creating mirror images of another Salesforce org's data, and because it was so easy to get XML copies of the official definitions of those fields as they existed in the "source" org, there had to be a way to use code to build those fields in the "target" org. I wanted to use code and a text editor because I wasn't trying to completely replicate the other Salesforce org - I had some "pruning" to do to get down to those 100-some items to create in my org and strip them of any inter-dependencies I didn't intend to mirror.

Indeed, there is a way. Here's how I did it.

---

Firstly, every time I needed the XML definition of objects & their custom fields from the central-university Salesforce org, I built an XML file called package.xml that looked something like this, uploaded it to Workbench while logged into that org, and saved the "{ObjectAPIName}.object" files in the "objects" folder in the resulting ZIP file.

<?xml version="1.0" encoding="UTF-8"?>
<Package xmlns="http://soap.sforce.com/2006/04/metadata">
    <fullName>GetObjectsPackage</fullName>
    <types>
        <members>Contact</members>
  <members>Application__c</members>
  <members>Test_Score__c</members>
  <members>App_Checklist_Document__c</members>
  <members>Recommendation__c</members>
  <members>Previous_Education__c</members>
  <members>Work_History__c</members>
        <name>CustomObject</name>
    </types>
    <version>36.0</version>
</Package>

---

Next, I wanted to throw away all the "junk" in those "{ObjectAPIName}.object" files from the central-university Salesforce org and extract only the contents of their "<fields>" tags.

The following Python code loops through a folder full of "{ObjectAPIName}.object" files and creates an "extracts" subfolder containing "{ObjectAPIName}-fields.xml" files showing just the fields I'm interested in (defined in ordinary string lists at the top of the code).

import os, fnmatch
from xml.dom import minidom

dumppath = 'C:\\SOMEPATH\\dumpeddata\\'
objapis = [f.rstrip('object').rstrip('.') for f in fnmatch.filter(os.listdir(dumppath), '*.object')]

objFieldsOfInterest = {
                        'Contact' : ['FirstName','LastName','Birthdate','Citizenship__c'],
                        'Application__c' : ['AppDate__c','AppStatus__c'],
                        'Test_Score__c' : ['Test_Score__c','Test_Type__c','Test_Date__c'],
                        'App_Checklist_Document__c' : ['Document_Type__c','Received_Date__c'],
                        'Recommendation__c' : ['Recommender_Last_Name__c','Recommender_First_Name__c','Relationship__c','Document_Status__c'],
                        'Previous_Education__c' : ['Document_Status__c','School_Name__c','Level__c','Graduation_Date__c'],
                        'Work_History__c' : ['Employer__c','Job_Title__c','Start_Date__c','End_Date__c']
                        }

objsAndDumpedXMLFields = {}
objsAndDumpedFieldNames = {}
objsAndXMLFieldsOfInterest = {}

# Fill the outer-level "object" dicts with XML and strings + write the data to disk
for o in objapis:
    objsAndDumpedXMLFields[o] = minidom.parse(dumppath+o+'.object').getElementsByTagName('fields')
    objsAndDumpedFieldNames[o] = [x.getElementsByTagName('fullName')[0].firstChild.nodeValue for x in objsAndDumpedXMLFields[o]]
    for f in objsAndDumpedXMLFields[o]:
        if o in objFieldsOfInterest and f.getElementsByTagName('fullName')[0].firstChild.nodeValue in objFieldsOfInterest[o]:
            #print(f.getElementsByTagName('fullName')[0].firstChild.nodeValue)
            if o not in objsAndXMLFieldsOfInterest:
                objsAndXMLFieldsOfInterest[o] = []
            objsAndXMLFieldsOfInterest[o].append(f)

# Write "objsAndXMLFieldsOfInterest" out to temporary files
if len(objsAndXMLFieldsOfInterest) > 0:
    if not os.path.exists(dumppath+'extracts\\'): os.makedirs(dumppath+'extracts\\')
    for filteredo in objsAndXMLFieldsOfInterest:
        with open(dumppath+'extracts\\'+filteredo+'-fields.xml', 'w') as outfile:
            xmlstrlist = []
            for elem in objsAndXMLFieldsOfInterest[filteredo]:
                xmlstrlist.append(elem.toxml())
            outfile.write('<?xml version="1.0" encoding="UTF-8"?>'+'\n'+'<rootnode>'+'\n'+'    '+'\n    '.join(xmlstrlist)+'\n'+'</rootnode>')

---

Cobbling together a folder full of "{ObjectAPIName}.object" files to upload into our Salesforce org was a pretty manual job in Notepad++.

I think I actually built the 6 custom objects by hand using Salesforce's normal web interface, then exported those ".object" files using Workbench as described above, and stripped pretty much all XML out of them (leaving only a few tags that seemed to be mandatory, such as "<sharingModel>," "<label>," "<nameField>," & "<pluralLabel>").

Then I re-filled those skeletons with the "<fields>" tags I'd extracted in the previous step (sometimes with some data altered - for example, I turned big picklists of countries into 100-character text fields to avoid having to keep picklists in sync, and I renamed "Contact.FirstName" from the extracted fields to "Contact.OtherDatabase_First_Name__c" for upload into our org).

---

Once I'd built such a folder full of "{ObjectAPIName}.object" files to upload into our Salesforce org, I had to put them inside a properly-formatted "package" that also included permissions for the new objects & fields.

To make this "package," I ran the following Python code (note that my permissions are pretty simple - you might need more code for more complex permissions):

import os
from xml.dom import minidom
from xml.etree.ElementTree import Element, SubElement, tostring
import itertools


oldobjectxmlspath = 'C:\\WhereIWasBuildingMyObjectFiles\\'

pkgpath = 'C:\\SomePath\\mynewpackage\\'
pkgfn = 'package.xml'
objpath = pkgpath+'objects\\'
prfpath = pkgpath+'profiles\\'
profiletypeswhocanediteverything = ['Admin','Admissions','Records Management']

objapis = None # Not really necessary, but this helps me keep track of outer-level variables
objswithfieldnames = {}
numobjswithfieldnames = 0
pkgroot = None

# Make the paths if they don't exist
if not os.path.exists(pkgpath): os.makedirs(pkgpath)
if not os.path.exists(objpath): os.makedirs(objpath)
if not os.path.exists(prfpath): os.makedirs(prfpath)

# Copy in any data we want to work with (COMMENT THIS OUT IF DATA IS ALREADY IN PLACE)
from shutil import copyfile
for i in os.listdir(oldobjectxmlspath):
    if not os.path.exists(objpath+i): copyfile(oldobjectxmlspath+i, objpath+i)

# Set "objapis" outer-level variable
objapis = [f.rstrip('object').rstrip('.') for f in os.listdir(objpath)] # Not sure why I can't just strip '.object'

# Set "objfieldnamesdict" outer-level variable
for f in objapis:
    objfields = [f.getElementsByTagName('fullName')[0].firstChild.nodeValue for f in minidom.parse(objpath+f+'.object').getElementsByTagName('fields')]
    if len(objfields) > 0:
        objswithfieldnames[f] = objfields

# Set numobjswithfieldnames outer-level variable for quick-ref later
numobjswithfieldnames = len(list(itertools.chain.from_iterable(objswithfieldnames.values())))


# Make our own "prettify" function since our IDE doesn't seem to have ElementPrettify in it
def prettify(elem):
    #Return a pretty-printed XML string for the Element.
    rough_string = tostring(elem, 'utf-8')
    reparsed = minidom.parseString(rough_string)
    return reparsed.toprettyxml(indent="\t")

# Build the XML for "package.xml" & for the can-edit-all-fields ".profile" files and write them to disk
if len(objapis) > 0:
    # "package.xml" file
    pkgroot = Element('Package')
    pkgroot.set('xmlns', 'http://soap.sforce.com/2006/04/metadata')
    SubElement(pkgroot, 'fullName').text = 'PyPkg'
    objTypesSE = SubElement(pkgroot, 'types')
    for o in objapis:
        SubElement(objTypesSE, 'members').text = o
    SubElement(objTypesSE, 'name').text = 'CustomObject'
    if numobjswithfieldnames > 0:
        fieldTypesSE = SubElement(pkgroot, 'types')
        for o in objswithfieldnames:
            for f in objswithfieldnames[o]:
                SubElement(fieldTypesSE, 'members').text = o+'.'+f
        SubElement(fieldTypesSE, 'name').text = 'CustomField'
    if len(profiletypeswhocanediteverything) > 0:
        profTypesSE = SubElement(pkgroot, 'types')
        for p in profiletypeswhocanediteverything:
            SubElement(profTypesSE, 'members').text = p
        SubElement(profTypesSE, 'name').text = 'Profile'
    SubElement(pkgroot, 'version').text = '36.0'
    with open(pkgpath+pkgfn, 'w') as pkgfile:
        pkgfile.write(prettify(pkgroot))
    
    # "profile\_____.profile" files for profiles who can edit all the fields in question:
    if len(profiletypeswhocanediteverything) > 0:
        for p in profiletypeswhocanediteverything:
            pfroot = Element('Profile')
            pfroot.set('xmlns', 'http://soap.sforce.com/2006/04/metadata')
            if numobjswithfieldnames > 0:
                for o in objswithfieldnames:
                    for f in objswithfieldnames[o]:
                        fieldPermSE = SubElement(pfroot, 'fieldPermissions')
                        SubElement(fieldPermSE, 'editable').text = 'true'
                        SubElement(fieldPermSE, 'field').text = o+'.'+f
                        SubElement(fieldPermSE, 'hidden').text = 'false'
                        SubElement(fieldPermSE, 'readable').text = 'true'
            for o in objapis:
                oPermSE = SubElement(pfroot, 'objectPermissions')
                SubElement(oPermSE, 'allowCreate').text = 'true'
                SubElement(oPermSE, 'allowDelete').text = 'true'
                SubElement(oPermSE, 'allowEdit').text = 'true'
                SubElement(oPermSE, 'allowRead').text = 'true'
                SubElement(oPermSE, 'modifyAllRecords').text = 'true'
                SubElement(oPermSE, 'object').text = o
                SubElement(oPermSE, 'viewAllRecords').text = 'true'
            with open(prfpath+p+'.profile', 'w') as pffile:
                pffile.write(prettify(pfroot))

But I wasn't quite done building the package after I ran the Python.

When the package includes objects that have a master-detail relationship to another object, the "detail" object's field pointing to the "master" MUST be included in its ".object" file and in "package.xml," but MUST NOT be mentioned in the ".profile" files. So next, I had to go through the contents of "C:\SomePath\mynewpackage\profile\" and strip the "<fieldPermissions>...</fieldPermissions>" blocks for those fields out of the ".profile" files. (I just did this for one ".profile" file in Notepad++, then copied/pasted the file's entire contents into the other ".profile" files, since they were all identical except in filename.)

I then zipped up "C:\SomePath\mynewpackage" into "mynewpackage.zip," went to Workbench while logged into our Salesforce org, and uploaded the package.
(It's best to turn on "Check Only" at first, and deploys to production orgs also require "Rollback On Error" checked and "Run Tests" to be set to "RunLocalTests".)

That's it! Workbench will tell you how the data push to your org went, and if things went well, go browse around your object definitions in the normal Salesforce web interface - everything should be there! You'll still have to add the fields to page layouts and such by hand (each new custom object will have a mostly-blank default page layout), but they will be there, visible to Jitterbit, data loading tools, Workbench, etc. (as long as it's logged in as someone who has permission to the objects/fields).

---

Finally, a few lessons I learned along the way in building my "packages" are:

1. If I include an object in a package and simply don't mention its existing fields, they will be left alone.
   • This means I can say, "Oops, I forgot those 5 fields!" later on and build a whole package just around adding them to our Salesforce org. It's faster than building those 5 fields by hand once all this infrastructure is in place.
2. If I include a profile in a package and simply don't mention its permissions on objects/fields, they will be left alone.
3. If an object/field already exists (matching via API name), anything that conflicts with its existing settings that I put into the package will overwrite that object's/field's settings if possible, and error out if not.
4. It's hard to write perfect Salesforce-Metadata-API-package XML the first time, so you might want a faster way to repeatedly "deploy" your packages (after getting error messages) than ZIPping+Workbench provides. Read more here about installing and using a tool for this purpose called ANT.