Thanks to StackExchange, I got the help I needed when I last posted about creating custom objects and fields with code.
My department at a university bought a Salesforce org 5 years ago. That's the one I'm the sysadmin for. We meant it to fill a lot of gaps left for our department by the university's central ERP database (Banner) - such as Banner not handling week-long "courses" very well, Banner not handling "deans' corporate rolodexes" very well, Banner not playing well with modern mass-emailing/texting/etc. software, etc.
Our university's central IT department just bought a Salesforce org about 1 year ago. We're not sure yet exactly how widely it will be used, but for starters, it's replacing our home-coded web-based application system (which used to dump straight into our ERP).
Because of that limited scope, it really doesn't yet fill any of the "holes" in the central ERP's offerings that my department bought its own Salesforce org for. Which means we're leaving both of them live.
And now instead of just worrying about integrating a daily dump from the ERP into our Salesforce org, I also get to set up a daily dump from the new central-Salesforce into our Salesforce.
Our typical pattern for "daily dumps" is "dump the data where it can't hurt anything, and move it where it belongs by trigger/workflow/process afterwards."
So, for example, "First Name" from Banner dumps into Contact.Banner_First_Name__c - and then is post-processed to fill out Contact.FirstName if there isn't already something there. (List View reports help us find & correct cases where the two fields have different values.)
I was about to dump 20 never-before-dumped fields' worth of data into Contact and about 6 objects (at 4-20 fields apiece) from yet another database into our Salesforce, so I had a lot of "creating objects & fields" to do before I could even get started setting up the Jitterbit feed between them.
What really bugged me was the idea that because I was essentially creating mirror images of another Salesforce org's data, and because it was so easy to get XML copies of the official definitions of those fields as they existed in the "source" org, there had to be a way to use code to build those fields in the "target" org. I wanted to use code and a text editor because I wasn't trying to completely replicate the other Salesforce org - I had some "pruning" to do to get down to those 100-some items to create in my org and strip them of any inter-dependencies I didn't intend to mirror.
Indeed, there is a way. Here's how I did it.
Firstly, every time I needed the XML definition of objects & their custom fields from the central-university Salesforce org, I built an XML file called package.xml that looked something like this, uploaded it to Workbench while logged into that org, and saved the "{ObjectAPIName}.object" files in the "objects" folder in the resulting ZIP file.
<?xml version="1.0" encoding="UTF-8"?>
<Package xmlns="http://soap.sforce.com/2006/04/metadata">
<fullName>GetObjectsPackage</fullName>
<types>
<members>Contact</members>
<members>Application__c</members>
<members>Test_Score__c</members>
<members>App_Checklist_Document__c</members>
<members>Recommendation__c</members>
<members>Previous_Education__c</members>
<members>Work_History__c</members>
<name>CustomObject</name>
</types>
<version>36.0</version>
</Package>
Next, I wanted to throw away all the "junk" in those "{ObjectAPIName}.object" files from the central-university Salesforce org and extract only the contents of their "<fields>" tags.
The following Python code loops through a folder full of "{ObjectAPIName}.object" files and creates an "extracts" subfolder containing "{ObjectAPIName}-fields.xml" files showing just the fields I'm interested in (defined in ordinary string lists at the top of the code).
import os, fnmatch
from xml.dom import minidom
dumppath = 'C:\\SOMEPATH\\dumpeddata\\'
objapis = [f.rstrip('object').rstrip('.') for f in fnmatch.filter(os.listdir(dumppath), '*.object')]
objFieldsOfInterest = {
'Contact' : ['FirstName','LastName','Birthdate','Citizenship__c'],
'Application__c' : ['AppDate__c','AppStatus__c'],
'Test_Score__c' : ['Test_Score__c','Test_Type__c','Test_Date__c'],
'App_Checklist_Document__c' : ['Document_Type__c','Received_Date__c'],
'Recommendation__c' : ['Recommender_Last_Name__c','Recommender_First_Name__c','Relationship__c','Document_Status__c'],
'Previous_Education__c' : ['Document_Status__c','School_Name__c','Level__c','Graduation_Date__c'],
'Work_History__c' : ['Employer__c','Job_Title__c','Start_Date__c','End_Date__c']
}
objsAndDumpedXMLFields = {}
objsAndDumpedFieldNames = {}
objsAndXMLFieldsOfInterest = {}
# Fill the outer-level "object" dicts with XML and strings + write the data to disk
for o in objapis:
objsAndDumpedXMLFields[o] = minidom.parse(dumppath+o+'.object').getElementsByTagName('fields')
objsAndDumpedFieldNames[o] = [x.getElementsByTagName('fullName')[0].firstChild.nodeValue for x in objsAndDumpedXMLFields[o]]
for f in objsAndDumpedXMLFields[o]:
if o in objFieldsOfInterest and f.getElementsByTagName('fullName')[0].firstChild.nodeValue in objFieldsOfInterest[o]:
#print(f.getElementsByTagName('fullName')[0].firstChild.nodeValue)
if o not in objsAndXMLFieldsOfInterest:
objsAndXMLFieldsOfInterest[o] = []
objsAndXMLFieldsOfInterest[o].append(f)
# Write "objsAndXMLFieldsOfInterest" out to temporary files
if len(objsAndXMLFieldsOfInterest) > 0:
if not os.path.exists(dumppath+'extracts\\'): os.makedirs(dumppath+'extracts\\')
for filteredo in objsAndXMLFieldsOfInterest:
with open(dumppath+'extracts\\'+filteredo+'-fields.xml', 'w') as outfile:
xmlstrlist = []
for elem in objsAndXMLFieldsOfInterest[filteredo]:
xmlstrlist.append(elem.toxml())
outfile.write('<?xml version="1.0" encoding="UTF-8"?>'+'\n'+'<rootnode>'+'\n'+' '+'\n '.join(xmlstrlist)+'\n'+'</rootnode>')
Cobbling together a folder full of "{ObjectAPIName}.object" files to upload into our Salesforce org was a pretty manual job in Notepad++.
I think I actually built the 6 custom objects by hand using Salesforce's normal web interface, then exported those ".object" files using Workbench as described above, and stripped pretty much all XML out of them (leaving only a few tags that seemed to be mandatory, such as "<sharingModel>," "<label>," "<nameField>," & "<pluralLabel>").
Then I re-filled those skeletons with the "<fields>" tags I'd extracted in the previous step (sometimes with some data altered - for example, I turned big picklists of countries into 100-character text fields to avoid having to keep picklists in sync, and I renamed "Contact.FirstName" from the extracted fields to "Contact.OtherDatabase_First_Name__c" for upload into our org).
Once I'd built such a folder full of "{ObjectAPIName}.object" files to upload into our Salesforce org, I had to put them inside a properly-formatted "package" that also included permissions for the new objects & fields.
To make this "package," I ran the following Python code (note that my permissions are pretty simple - you might need more code for more complex permissions):
import os
from xml.dom import minidom
from xml.etree.ElementTree import Element, SubElement, tostring
import itertools
oldobjectxmlspath = 'C:\\WhereIWasBuildingMyObjectFiles\\'
pkgpath = 'C:\\SomePath\\mynewpackage\\'
pkgfn = 'package.xml'
objpath = pkgpath+'objects\\'
prfpath = pkgpath+'profiles\\'
profiletypeswhocanediteverything = ['Admin','Admissions','Records Management']
objapis = None # Not really necessary, but this helps me keep track of outer-level variables
objswithfieldnames = {}
numobjswithfieldnames = 0
pkgroot = None
# Make the paths if they don't exist
if not os.path.exists(pkgpath): os.makedirs(pkgpath)
if not os.path.exists(objpath): os.makedirs(objpath)
if not os.path.exists(prfpath): os.makedirs(prfpath)
# Copy in any data we want to work with (COMMENT THIS OUT IF DATA IS ALREADY IN PLACE)
from shutil import copyfile
for i in os.listdir(oldobjectxmlspath):
if not os.path.exists(objpath+i): copyfile(oldobjectxmlspath+i, objpath+i)
# Set "objapis" outer-level variable
objapis = [f.rstrip('object').rstrip('.') for f in os.listdir(objpath)] # Not sure why I can't just strip '.object'
# Set "objfieldnamesdict" outer-level variable
for f in objapis:
objfields = [f.getElementsByTagName('fullName')[0].firstChild.nodeValue for f in minidom.parse(objpath+f+'.object').getElementsByTagName('fields')]
if len(objfields) > 0:
objswithfieldnames[f] = objfields
# Set numobjswithfieldnames outer-level variable for quick-ref later
numobjswithfieldnames = len(list(itertools.chain.from_iterable(objswithfieldnames.values())))
# Make our own "prettify" function since our IDE doesn't seem to have ElementPrettify in it
def prettify(elem):
#Return a pretty-printed XML string for the Element.
rough_string = tostring(elem, 'utf-8')
reparsed = minidom.parseString(rough_string)
return reparsed.toprettyxml(indent="\t")
# Build the XML for "package.xml" & for the can-edit-all-fields ".profile" files and write them to disk
if len(objapis) > 0:
# "package.xml" file
pkgroot = Element('Package')
pkgroot.set('xmlns', 'http://soap.sforce.com/2006/04/metadata')
SubElement(pkgroot, 'fullName').text = 'PyPkg'
objTypesSE = SubElement(pkgroot, 'types')
for o in objapis:
SubElement(objTypesSE, 'members').text = o
SubElement(objTypesSE, 'name').text = 'CustomObject'
if numobjswithfieldnames > 0:
fieldTypesSE = SubElement(pkgroot, 'types')
for o in objswithfieldnames:
for f in objswithfieldnames[o]:
SubElement(fieldTypesSE, 'members').text = o+'.'+f
SubElement(fieldTypesSE, 'name').text = 'CustomField'
if len(profiletypeswhocanediteverything) > 0:
profTypesSE = SubElement(pkgroot, 'types')
for p in profiletypeswhocanediteverything:
SubElement(profTypesSE, 'members').text = p
SubElement(profTypesSE, 'name').text = 'Profile'
SubElement(pkgroot, 'version').text = '36.0'
with open(pkgpath+pkgfn, 'w') as pkgfile:
pkgfile.write(prettify(pkgroot))
# "profile\_____.profile" files for profiles who can edit all the fields in question:
if len(profiletypeswhocanediteverything) > 0:
for p in profiletypeswhocanediteverything:
pfroot = Element('Profile')
pfroot.set('xmlns', 'http://soap.sforce.com/2006/04/metadata')
if numobjswithfieldnames > 0:
for o in objswithfieldnames:
for f in objswithfieldnames[o]:
fieldPermSE = SubElement(pfroot, 'fieldPermissions')
SubElement(fieldPermSE, 'editable').text = 'true'
SubElement(fieldPermSE, 'field').text = o+'.'+f
SubElement(fieldPermSE, 'hidden').text = 'false'
SubElement(fieldPermSE, 'readable').text = 'true'
for o in objapis:
oPermSE = SubElement(pfroot, 'objectPermissions')
SubElement(oPermSE, 'allowCreate').text = 'true'
SubElement(oPermSE, 'allowDelete').text = 'true'
SubElement(oPermSE, 'allowEdit').text = 'true'
SubElement(oPermSE, 'allowRead').text = 'true'
SubElement(oPermSE, 'modifyAllRecords').text = 'true'
SubElement(oPermSE, 'object').text = o
SubElement(oPermSE, 'viewAllRecords').text = 'true'
with open(prfpath+p+'.profile', 'w') as pffile:
pffile.write(prettify(pfroot))
But I wasn't quite done building the package after I ran the Python.
When the package includes objects that have a master-detail relationship to another object, the "detail" object's field pointing to the "master" MUST be included in its ".object" file and in "package.xml," but MUST NOT be mentioned in the ".profile" files. So next, I had to go through the contents of "C:\SomePath\mynewpackage\profile\" and strip the "<fieldPermissions>...</fieldPermissions>" blocks for those fields out of the ".profile" files. (I just did this for one ".profile" file in Notepad++, then copied/pasted the file's entire contents into the other ".profile" files, since they were all identical except in filename.)
I then zipped up "C:\SomePath\mynewpackage" into "mynewpackage.zip," went to Workbench while logged into our Salesforce org, and uploaded the package.
(It's best to turn on "Check Only" at first, and deploys to production orgs also require "Rollback On Error" checked and "Run Tests" to be set to "RunLocalTests".)
That's it! Workbench will tell you how the data push to your org went, and if things went well, go browse around your object definitions in the normal Salesforce web interface - everything should be there! You'll still have to add the fields to page layouts and such by hand (each new custom object will have a mostly-blank default page layout), but they will be there, visible to Jitterbit, data loading tools, Workbench, etc. (as long as it's logged in as someone who has permission to the objects/fields).
Finally, a few lessons I learned along the way in building my "packages" are:
- If I include an object in a package and simply don't mention its existing fields, they will be left alone.
- This means I can say, "Oops, I forgot those 5 fields!" later on and build a whole package just around adding them to our Salesforce org. It's faster than building those 5 fields by hand once all this infrastructure is in place.
- If I include a profile in a package and simply don't mention its permissions on objects/fields, they will be left alone.
- If an object/field already exists (matching via API name), anything that conflicts with its existing settings that I put into the package will overwrite that object's/field's settings if possible, and error out if not.
- It's hard to write perfect Salesforce-Metadata-API-package XML the first time, so you might want a faster way to repeatedly "deploy" your packages (after getting error messages) than ZIPping+Workbench provides. Read more here about installing and using a tool for this purpose called ANT.
No comments:
Post a Comment