Marc authority multitenant import

Owned by Roman_Fedynyshyn
Last updated: Feb 25, 2022
4 min read



PERF-210 - Test Import of MARC Authorities IN REVIEW

UIDATIMP-1033 - Performance Test: Data Import > Create MARC authority records IN PROGRESS


Overview 

This document contains the results of testing Data Import MARC Authorities in pre-lotus to detect performance trends. 

This document contains result of testing with multiple tenants concurrently with and without Check-in-Check-out process on the background;

Summary 




  • Differences in times of execution between stand alone job and multitenant 4-5 times (1 min 14 s for 5 k and 5 min 30 s for 5k in multitenant, 2 minutes to 7 minutes in 10k )
  • It’s not so stable when it comes to big files we saw a lot of DB timeouts  (so maybe it will be a good idea to extend timeouts or investigate them deeper)
  • We can see memory usage growing from test to test it might be a sign of memory leak. PERF-223 - Getting issue details... STATUS


Multitenant testing with CICO

  • CICO has no errors during test set
  • Times for DI grew even more +2-3 minutes and +20 minutes for 50K
  • 50K failed (completed with errors) on same tenant as CICO was working on.

Multitenant Marc authority imports 2 tenants

Test Durations
Tenant 1 Tenant 21 tenant (Controlled)
5 K2 min 50s2 mn 51 s1 min 14 s
10 K5 min 23 s5 min 14 s2 min 32 s
25 K14 min 03 s12 min 53 s6 min 
50 K24 min 40 s24 min 37 s12 min

Resources usage








Multitenant Marc authority imports 3 tenants

To test this we've used three tenants with shared resources.


Test Durations
Tenant 1 Tenant 2Tenant 31 tenant (Controlled)
5 K5 min 56 s5 min 37 s5 min 29 s1 min 14 s
10 K7 min 48 s10 min 44 s11 min 36 s2 min 32 s
25 K21 min 26 s25 min 37 s40 min 13 s6 min 
50 K41 min 29 s41 min 45 s42 min 15 s12 min

Resources usage

CPU has predicted behavior with one spike at the beginning of the test



As you can see there is growing memory usage from test to test it might be a sign of memory leak. 

COU usage of EC2 instances are less than 30%. 






50K









Multitenant Marc authority imports 3 tanants + CICO 

Test Durations
Tenant 1 Tenant 2Tenant 3
5 K6 min 33 s6 min 38 s6 min 12 s
10 K9 min 9 s9 min 52 s9 min 22 s
25 K23 min 48 s23 min 36 s23 min 26 s
50 K1 hr 11 min 34 s1 hr 11 min 30s1 hr 13 min





Resources usage







Check-In-Check-Out statistics 





Marc authority + CICO + data import (create srs records, instances, holdings, items (Ptf Create 2 job profile))


Test Durations
Tenant 1 1 tenant (Controlled)
5 K15 min 24 s1 min 14 s
10 K17 min 27 s2 min 32 s
25 K35 min 21 s6 min 
50 K1 hr 4 min 14 s12 min


Resources usage