The Importance of Prototyping with Production Data
The Importance of Prototyping with Production Data
The performance, response time and "number of records returned" problems associated with name search relate, among other things, to the volume of data in the database and the skew of the distribution of names.
The reliability problems associated with name search relate, among other things, to the quality and make-up of the data being searched.
Name searches should be tested, or the results evaluated, by expert users who can feedback reliable information to the designer.
Normal test data cannot illustrate these volume & quality related problems. A name search system may pass design and acceptance testing but fail miserably in production for this reason. For example:
a customer search which tests successfully on the 500 record employee file, is no test of how it will perform on the 5 million record customer file;
a search which finds "TEST MICKY MOUSE, XXXXXXX XXXXXXXXXXXX" or "THIS IS A VERY LONG NAME FOR TESTING", is no test of whether it will find "EYAL LEBEDINSKY", "ABUL MOHD AZIZ RAMAN" or "BILLY SAY LIM HO";
a test search which uses the full name as search criteria is no good if the user ultimately only has a surname and an initial to search with.
Therefore, all but the initial functional testing of name search applications should be carried out on Production data and Production volumes. This also means that the data used to search with must also be appropriate for the production scenario.
If the Production data is loaded into a development or test environment, care should be taken to not deduce "production" response times from these environments, as the production system environment may be very different. It may be possible to monitor the average number of records returned from a search and extrapolate the average record access time to the production scenario, but this requires some careful investigation.
This chapter provides a background to why Informatica’s approach to identity search and matching supports strong multi-national systems.