The UM-SIG meeting time overlaps with the regular OLE implementers call; we may want to look at this
User bulk loading discussion
Begin discussing requirements for patron bulk loading.
István proposed potential v1 for bulk loading
Import files with fixed columns
Simple delimited file support (eg. CSV)
File validation support, error reporting
Feedback from SMEs:
Need to be able to point to directory and update the file daily
Need good reporting (can be 70 k records in our patron file every day)
Any sane format would be fine (json, xml, csv) because there will need to be a transformation made from PeopleSoft SIS anyway
Duke gets file from LDAP
Duke also does a lot of data transformation for their feed - if this could be done in FOLIO, that would be really nice (but it's currently done outside of their ILS)
Updating records instead of overwriting is very important
Definitely don’t want duplicate records
Karen and Chris: If Folio can ingest a feed file in a smart way, that would be better than having to do the transformations outside of the system
The trick is going to be that the format and the feed will be different for each library
Chris: If we have a defined standard that FOLIO is expecting then it might be possible to get a PeopleSoft export in the proper formatting and avoid the need to do additional intermediary processing. I could just
Karen: We map patron types by the org unit by which they get paid and (there are many)
Record IDs for external system:
We’ve got a couple of identifiers we can bring in and want to bring in
Internal identifier from Peoplesoft
Network id is what we’ll want to put into the userid field
We’d need to determine which of the identifiers is used for matching
Can we all agree that the external systems id is what we match on? Or does this need to be a library configuration?
Error reporting functionality:
How should system behave if records aren’t able to be updated?
Chris would want to see a report sent out with X records couldn’t be loaded
Dump unloaded records so you could look at them and possibly load them
This would actually be handy to have that manual upload capability in the UI to do that
Chris can get Istvan the output that his current system provides, but it’s an example of what we have now
Circ folks might want some positive reporting (e.g. 70 k successfully loaded today)
This would allow you to spot if the feed file is suddenly lower than it was yesterday
What kind of validation is done in systems today and what happens when invalid data is encountered? Is the record updated and the invalid data ignored? Is the entire record update ignored?
Chris: We don’t know how much validation is done - we might just accept invalid data
Chris will bundle up a bunch of info regarding how things are now done
In current systems, there is a distinction between patrons and staff/operators
Only patron records come in via feed
Staff/operator record are created by hand and permissions are added then
Could operator data come from the feed?
Active directory could be used for this
But there is not really a lot of staff turnover – administering that manually as we do now is okay
Permissions would be difficult to be loaded
Future thinking around user data
Duke would like to be able to use grouper etc
We have a pie in the sky idea where we would load minimal data for the user and hit the LDAP server to get the rest of the data so we don’t have to store the additional info (addresses etc)
When we have a new patron, being able to scan the barcode to have the patron data loaded
There is the opportunity to use a different approach – LDAP
But how many libraries are ready for this?
In current systems, patron images aren’t stored or displayed
Don't know how we would link out to them
Not sure it's necessary to have photos in FOLIO - patron hands over their cards which have their photos. Photo on screen isn’t all that useful. May even be privacy issues.
Lehigh would like to see real-time, transactional API for user updates
If bulk loader is iterating across that transactional API, cool!