Assessment ratings
Table
Rating categories: A, B, C, D, F where A is the highest
| Category | Rating | Comments |
|---|---|---|
| Performance | C | Reached POC performance at "create" (rated with B) and low performance for "update" (rated at D) |
| Stability/Reliability | D | Has issues with imports stability. To be addressed with highest priority |
| Scalability | A | The solution scales well |
| Architecture | B | No major changes required. A number of aspects can be optimized |
| Code quality | C | Lacking test coverage, a number of functional bugs to be fixed |
OVERALL RATING: C (as stability/reliability issues have highest priority)
Categories
Performance
Original estimate set was 6 min/ 10k for instances only creation.
From original POC document
We have successfully imported files of different sizes using this PoC. A simple Data-Import job profile was used. It contains the “Create Instance” action with the default mapping without any lookup and matching. The profile does not contain any additional actions to create entities like “Holdings” or “Items”.
Current state is 29min / 10k instances, holdings and items + SRS records created (for original setup - 2 nodes per each DI module except mod-date-import).
Further improvements can be done to increase performance for complex profiles.
Impacts other processes during large imports.
Stability/Reliability
Is the solution stable?
The solution allows to perform stable imports up to 10k and has issues with higher amounts of data.
- INSERT: Stable from 10k to 25k. 25k+ has stability issues
- UPDATE: Stable up to 5k, and has issues over 5k
Are there issues with data integrity?
Several issues for duplicating data are found and currently being fixed.
Eventually not all records are processed. In investigation.
Repeatability of imports?
There is a performance degradation for consequential imports. In investigation.
Scalability
How does the system react for system resources increase/decrease and nodes manipulation
Vertically: the solution scales well upon increasing available resources (see: PTF - DI testing for Cornell (Iris hotfixes))
Horizontally: the solution reaches up to 50% effectiveness increase upon increasing number of processing nodes (1 node 10k import: 45min, 2 nodes 10k import: 29 min – scaling applied to SRS, SRM, Inventory, Inventory-storage)
Needs more testing for higher numbers (2+ nodes).
Possible errors if scaling Kafka without restarting consumers (needs to be tested)
Architecture
How good is the architecture?
The solution has solid architecture, allowing to solve and scale initial requirements.
How does the solution accept changes in business requirements?
Upon development there were several changes done to initial requirements which were implemented without major impact on solution architecture.
Are there known points to be changed to improve current state?
There are some optimizations and improvements planned for current solution. Overall the solution does not need any significant rework and only needs to be adjusted for better reliability and performance with local optimizations.
Code quality
How does the code solve business cases (number of bugs, tests, etc)
Current implementation solves multiple business cases including standard create/update MARC_Bib data-import flows, OCLC single record imports, quickMarc create/edit, import of MARC_Authority and Edifact records. However, 38 functional bugs were reported since R1 release, 32 of them are already fixed. Coverage by unit tests average to around 80%, but some of the modules lack sufficient coverage.