SourceRecordStorage tools for test data populating
Endpoint
To test the module, there is a special endpoint for loading test data into the module's storage. Ability to load MARC records supports the loading of both source data and already parsed records in JSON format.
This endpoint does not need any permissions. All records imported through this endpoint would be linked with the stubbed snapshot. Please see API documentation.
| Method | Description |
|---|---|
POST /populate-test-marc-records | Non-production endpoint into the mod-source-record-storage that allow the addition of MARC records for testing purposes. mod-source-record-storage have a dedicated stub snapshot to store such records. |
Schema
Endpoint works with a simple data structure for loading a bunch of source records.
MarcRecordCollection
| Field | Description |
|---|---|
| rawRecords | Array of raw records that contains raw data of MARC record and UUID |
RawRecord
| Field | Description |
|---|---|
| id | UUID for particular record. Endpoint don't generate a UUID for records. Required field |
| content | String representation of single MARC record. The content can be either raw data or already parsed data in JSON format |
Example
MarcRecordCollection in this example contains two records inside the array. First source record is in raw format and the second one is represented as JSON.
{
"rawRecords": [
{
"id": "88dfac11-1caf-4470-9ad1-d533f6360bdd",
"content": "01240cas a2200397 4500001000700000005001700007008004100024010001700065022001400082035002600096035002200122035001100144035001900155040004400174050001500218082001100233222004200244245004300286260004700329265003800376300001500414310002200429321002500451362002300476570002900499650003300528650004500561655004200606700004500648853001800693863002300711902001600734905002100750948003700771950003400808\u001E366832\u001E20141106221425.0\u001E750907c19509999enkqr p 0 a0eng d\u001E \u001Fa 58020553 \u001E \u001Fa0022-0469\u001E \u001Fa(CStRLIN)NYCX1604275S\u001E \u001Fa(NIC)notisABP6388\u001E \u001Fa366832\u001E \u001Fa(OCoLC)1604275\u001E \u001FdCtY\u001FdMBTI\u001FdCtY\u001FdMBTI\u001FdNIC\u001FdCStRLIN\u001FdNIC\u001E0 \u001FaBR140\u001Fb.J6\u001E \u001Fa270.05\u001E04\u001FaThe Journal of ecclesiastical history\u001E04\u001FaThe Journal of ecclesiastical history.\u001E \u001FaLondon,\u001FbCambridge University Press [etc.]\u001E \u001Fa32 East 57th St., New York, 10022\u001E \u001Fav.\u001Fb25 cm.\u001E \u001FaQuarterly,\u001Fb1970-\u001E \u001FaSemiannual,\u001Fb1950-69\u001E0 \u001Fav. 1- Apr. 1950-\u001E \u001FaEditor: C. W. Dugmore.\u001E 0\u001FaChurch history\u001FxPeriodicals.\u001E 7\u001FaChurch history\u001F2fast\u001F0(OCoLC)fst00860740\u001E 7\u001FaPeriodicals\u001F2fast\u001F0(OCoLC)fst01411641\u001E1 \u001FaDugmore, C. W.\u001Fq(Clifford William),\u001Feed.\u001E03\u001F81\u001Fav.\u001Fi(year)\u001E40\u001F81\u001Fa1-49\u001Fi1950-1998\u001E \u001Fapfnd\u001FbLintz\u001E \u001Fa19890510120000.0\u001E2 \u001Fa20141106\u001Fbm\u001Fdbatch\u001Felts\u001Fxaddfast\u001E \u001FlOLIN\u001FaBR140\u001Fb.J86\u001Fh01/01/01 N\u001E\u001D"
},
{
"id": "77dfac11-1caf-4470-9ad1-d533f6360bdd",
"content": "{ \"leader\": \"00508cjm a22001813 4500\", \"fields\": [ { \"001\": \"10062588\" }, { \"005\": \"20171013073237.0\" }, { \"007\": \"sd fsngnnmmneu\" }, { \"008\": \"170825s2017 xx nn n zxx d\" }, { \"024\": { \"subfields\": [ { \"a\": \"00190295755553\" }, { \"2\": \"gtin-14\" } ], \"ind1\": \"7\", \"ind2\": \" \" } }, { \"024\": { \"subfields\": [ { \"a\": \"190295755553\" } ], \"ind1\": \"1\", \"ind2\": \" \" } }, { \"035\": { \"subfields\": [ { \"a\": \"(OCoLC)1002130878\" } ], \"ind1\": \" \", \"ind2\": \" \" } }, { \"035\": { \"subfields\": [ { \"a\": \"10062588\" } ], \"ind1\": \" \", \"ind2\": \" \" } }, { \"040\": { \"subfields\": [ { \"a\": \"BTCTA\" }, { \"b\": \"eng\" }, { \"c\": \"BTCTA\" } ], \"ind1\": \" \", \"ind2\": \" \" } }, { \"100\": { \"subfields\": [ { \"a\": \"Rossi, Daniele\" } ], \"ind1\": \"1\", \"ind2\": \" \" } }, { \"245\": { \"subfields\": [ { \"a\": \"Saint-Saens: Organ Symphony and Carnival of The Animals\" } ], \"ind1\": \"0\", \"ind2\": \"0\" } }, { \"260\": { \"subfields\": [ { \"b\": \"Wea Corp\" }, { \"c\": \"2017.\" } ], \"ind1\": \" \", \"ind2\": \" \" } }, { \"948\": { \"subfields\": [ { \"a\": \"20171013\" }, { \"b\": \"m\" }, { \"d\": \"batch\" }, { \"e\": \"lts\" }, { \"x\": \"deloclcprefix\" } ], \"ind1\": \"2\", \"ind2\": \" \" } } ] }"
}
]
}
How to load test data to the server
cURL
To load large amounts of test data, you can use the cURL CLI tool. The first step is preparing a file with request body in JSON format.
request.json
request.json
{
"rawRecords": [
{
"id": "88dfac11-1caf-4470-9ad1-d533f6360bdd",
"content": "01240cas a2200397 4500001000700000005001700007008004100024010001700065022001400082035002600096035002200122035001100144035001900155040004400174050001500218082001100233222004200244245004300286260004700329265003800376300001500414310002200429321002500451362002300476570002900499650003300528650004500561655004200606700004500648853001800693863002300711902001600734905002100750948003700771950003400808\u001E366832\u001E20141106221425.0\u001E750907c19509999enkqr p 0 a0eng d\u001E \u001Fa 58020553 \u001E \u001Fa0022-0469\u001E \u001Fa(CStRLIN)NYCX1604275S\u001E \u001Fa(NIC)notisABP6388\u001E \u001Fa366832\u001E \u001Fa(OCoLC)1604275\u001E \u001FdCtY\u001FdMBTI\u001FdCtY\u001FdMBTI\u001FdNIC\u001FdCStRLIN\u001FdNIC\u001E0 \u001FaBR140\u001Fb.J6\u001E \u001Fa270.05\u001E04\u001FaThe Journal of ecclesiastical history\u001E04\u001FaThe Journal of ecclesiastical history.\u001E \u001FaLondon,\u001FbCambridge University Press [etc.]\u001E \u001Fa32 East 57th St., New York, 10022\u001E \u001Fav.\u001Fb25 cm.\u001E \u001FaQuarterly,\u001Fb1970-\u001E \u001FaSemiannual,\u001Fb1950-69\u001E0 \u001Fav. 1- Apr. 1950-\u001E \u001FaEditor: C. W. Dugmore.\u001E 0\u001FaChurch history\u001FxPeriodicals.\u001E 7\u001FaChurch history\u001F2fast\u001F0(OCoLC)fst00860740\u001E 7\u001FaPeriodicals\u001F2fast\u001F0(OCoLC)fst01411641\u001E1 \u001FaDugmore, C. W.\u001Fq(Clifford William),\u001Feed.\u001E03\u001F81\u001Fav.\u001Fi(year)\u001E40\u001F81\u001Fa1-49\u001Fi1950-1998\u001E \u001Fapfnd\u001FbLintz\u001E \u001Fa19890510120000.0\u001E2 \u001Fa20141106\u001Fbm\u001Fdbatch\u001Felts\u001Fxaddfast\u001E \u001FlOLIN\u001FaBR140\u001Fb.J86\u001Fh01/01/01 N\u001E\u001D"
},
{
"id": "77dfac11-1caf-4470-9ad1-d533f6360bdd",
"content": "{ \"leader\": \"00508cjm a22001813 4500\", \"fields\": [ { \"001\": \"10062588\" }, { \"005\": \"20171013073237.0\" }, { \"007\": \"sd fsngnnmmneu\" }, { \"008\": \"170825s2017 xx nn n zxx d\" }, { \"024\": { \"subfields\": [ { \"a\": \"00190295755553\" }, { \"2\": \"gtin-14\" } ], \"ind1\": \"7\", \"ind2\": \" \" } }, { \"024\": { \"subfields\": [ { \"a\": \"190295755553\" } ], \"ind1\": \"1\", \"ind2\": \" \" } }, { \"035\": { \"subfields\": [ { \"a\": \"(OCoLC)1002130878\" } ], \"ind1\": \" \", \"ind2\": \" \" } }, { \"035\": { \"subfields\": [ { \"a\": \"10062588\" } ], \"ind1\": \" \", \"ind2\": \" \" } }, { \"040\": { \"subfields\": [ { \"a\": \"BTCTA\" }, { \"b\": \"eng\" }, { \"c\": \"BTCTA\" } ], \"ind1\": \" \", \"ind2\": \" \" } }, { \"100\": { \"subfields\": [ { \"a\": \"Rossi, Daniele\" } ], \"ind1\": \"1\", \"ind2\": \" \" } }, { \"245\": { \"subfields\": [ { \"a\": \"Saint-Saens: Organ Symphony and Carnival of The Animals\" } ], \"ind1\": \"0\", \"ind2\": \"0\" } }, { \"260\": { \"subfields\": [ { \"b\": \"Wea Corp\" }, { \"c\": \"2017.\" } ], \"ind1\": \" \", \"ind2\": \" \" } }, { \"948\": { \"subfields\": [ { \"a\": \"20171013\" }, { \"b\": \"m\" }, { \"d\": \"batch\" }, { \"e\": \"lts\" }, { \"x\": \"deloclcprefix\" } ], \"ind1\": \"2\", \"ind2\": \" \" } } ] }"
}
]
}
After file is ready you can run curl command.
curl -w '\n' -X POST -D - \
-H "Content-type: application/json" \
-H "x-okapi-tenant: diku" \
-d @request.json \
http://localhost:9130/populate-test-marc-records
If all records are successfully saved on the server, the response will come with the status code 204.
The server will determine in what format the data came. If it is raw data, it will parse the records and save both the original and the parsed record in JSON format.
If the source record is already in JSON format, then the server will save this JSON as the raw record as well as the parsed one.
Java client generated by RMB
For using module's endpoints it provides generated by RMB client. This client is packaged into the lightweight jar. You can add mod-source-record-storage-client as a maven dependency.
<dependency> <groupId>org.folio</groupId> <artifactId>mod-source-record-storage-client</artifactId> <version>x.y.z</version> <type>jar</type> </dependency>
Where x.y.z - version of mod-source-record-storage.
SourceStorageClient is generated by RMB and provide methods for all modules endpoints described in the RAML file. So in this case we can use /populate-test-marc-records endpoint throughs this client.
Clients methods works with generated by RMB data classes based on json schemas.
mod-source-record-storage-client jar contains only generated by RMB DTOs and clients.
Example with sending request to the mod-source-record-storage for creating test records
// create client object with okapi url, tenant id and token
SourceStorageClient client = new SourceStorageClient("localhost", "diku", "token");
// create new records entity
TestMarcRecordsCollection testData = new TestMarcRecordsCollection()
.withRawRecords(Arrays.asList(new RawRecord()
.withId("88dfac11-1caf-4470-9ad1-d533f6360bdd")
.withContent("01240cas a2200397 4500001000700000005001700007008004100024010001700065022001400082035002600096035002200122035001100144035001900155040004400174050001500218082001100233222004200244245004300286260004700329265003800376300001500414310002200429321002500451362002300476570002900499650003300528650004500561655004200606700004500648853001800693863002300711902001600734905002100750948003700771950003400808\\u001E366832\\u001E20141106221425.0\\u001E750907c19509999enkqr p 0 a0eng d\\u001E \\u001Fa 58020553 \\u001E \\u001Fa0022-0469\\u001E \\u001Fa(CStRLIN)NYCX1604275S\\u001E \\u001Fa(NIC)notisABP6388\\u001E \\u001Fa366832\\u001E \\u001Fa(OCoLC)1604275\\u001E \\u001FdCtY\\u001FdMBTI\\u001FdCtY\\u001FdMBTI\\u001FdNIC\\u001FdCStRLIN\\u001FdNIC\\u001E0 \\u001FaBR140\\u001Fb.J6\\u001E \\u001Fa270.05\\u001E04\\u001FaThe Journal of ecclesiastical history\\u001E04\\u001FaThe Journal of ecclesiastical history.\\u001E \\u001FaLondon,\\u001FbCambridge University Press [etc.]\\u001E \\u001Fa32 East 57th St., New York, 10022\\u001E \\u001Fav.\\u001Fb25 cm.\\u001E \\u001FaQuarterly,\\u001Fb1970-\\u001E \\u001FaSemiannual,\\u001Fb1950-69\\u001E0 \\u001Fav. 1- Apr. 1950-\\u001E \\u001FaEditor: C. W. Dugmore.\\u001E 0\\u001FaChurch history\\u001FxPeriodicals.\\u001E 7\\u001FaChurch history\\u001F2fast\\u001F0(OCoLC)fst00860740\\u001E 7\\u001FaPeriodicals\\u001F2fast\\u001F0(OCoLC)fst01411641\\u001E1 \\u001FaDugmore, C. W.\\u001Fq(Clifford William),\\u001Feed.\\u001E03\\u001F81\\u001Fav.\\u001Fi(year)\\u001E40\\u001F81\\u001Fa1-49\\u001Fi1950-1998\\u001E \\u001Fapfnd\\u001FbLintz\\u001E \\u001Fa19890510120000.0\\u001E2 \\u001Fa20141106\\u001Fbm\\u001Fdbatch\\u001Felts\\u001Fxaddfast\\u001E \\u001FlOLIN\\u001FaBR140\\u001Fb.J86\\u001Fh01/01/01 N\\u001E\\u001D"),
new RawRecord()
.withId("77dfac11-1caf-4470-9ad1-d533f6360bdd")
.withContent("{ \"leader\": \"00508cjm a22001813 4500\", \"fields\": [ { \"001\": \"10062588\" }, { \"005\": \"20171013073237.0\" }, { \"007\": \"sd fsngnnmmneu\" }, { \"008\": \"170825s2017 xx nn n zxx d\" }, { \"024\": { \"subfields\": [ { \"a\": \"00190295755553\" }, { \"2\": \"gtin-14\" } ], \"ind1\": \"7\", \"ind2\": \" \" } }, { \"024\": { \"subfields\": [ { \"a\": \"190295755553\" } ], \"ind1\": \"1\", \"ind2\": \" \" } }, { \"035\": { \"subfields\": [ { \"a\": \"(OCoLC)1002130878\" } ], \"ind1\": \" \", \"ind2\": \" \" } }, { \"035\": { \"subfields\": [ { \"a\": \"10062588\" } ], \"ind1\": \" \", \"ind2\": \" \" } }, { \"040\": { \"subfields\": [ { \"a\": \"BTCTA\" }, { \"b\": \"eng\" }, { \"c\": \"BTCTA\" } ], \"ind1\": \" \", \"ind2\": \" \" } }, { \"100\": { \"subfields\": [ { \"a\": \"Rossi, Daniele\" } ], \"ind1\": \"1\", \"ind2\": \" \" } }, { \"245\": { \"subfields\": [ { \"a\": \"Saint-Saens: Organ Symphony and Carnival of The Animals\" } ], \"ind1\": \"0\", \"ind2\": \"0\" } }, { \"260\": { \"subfields\": [ { \"b\": \"Wea Corp\" }, { \"c\": \"2017.\" } ], \"ind1\": \" \", \"ind2\": \" \" } }, { \"948\": { \"subfields\": [ { \"a\": \"20171013\" }, { \"b\": \"m\" }, { \"d\": \"batch\" }, { \"e\": \"lts\" }, { \"x\": \"deloclcprefix\" } ], \"ind1\": \"2\", \"ind2\": \" \" } } ] }")));
// send request to mod-source-record-storage
client.postSourceStoragePopulateTestMarcRecords(testData, handler->{
if (handler.statusCode()==204){
// processing response
System.out.println("Records are successfully created.");
}
});