Organising data before automating your business

Automation is meant to save time. Yet after it goes live, employees sometimes end up with even more work: correcting wrong invoices, checking duplicate orders and explaining why a customer received a message with an outdated status. The cause often lies in the data that feeds the automation.

You do not need a perfect database to get started. You do need to know where the key information comes from, what it means and what the system should do when it is incomplete or contradictory. Below we show how to prepare data for automation without a months-long project to clean up the whole company.

The short answer: what should you prepare before automating?

Before you connect a form, a store, a CRM, accounting software or a spreadsheet into one workflow, settle five things:

• What data is needed to make a decision in the process.
• In which system each important piece of information is created and updated.
• How to recognise the same customer, product, order or case across different systems.
• Which values are correct, and what to do with a missing or wrong value.
• Who receives a case the automation cannot handle safely.

That is enough to build a sensible plan for the first rollout. Detailed clean-up is best started with the data that actually drives the automation.

Why does data matter so much?

Imagine a store where orders land in the sales system, invoices are created in a separate program and delivery information is updated in a spreadsheet. The company wants to issue invoices automatically and send customers notifications about their order status.

The idea is simple, but questions arise at once. Does “shipped” in the store mean the same as “dispatched” in the courier’s system? Where do you get the current address for the invoice? Are two entries with the same company name one customer? What do you do when an order has no VAT number (NIP), although the customer chose a company invoice?

If nobody settles the answers, the automation will run on accidental assumptions. It may send a message too early, issue a document with outdated data or stop without a clear message. That is why preparing data is part of designing the process, not a separate “technical tidy-up” for later.

Step 1. Map out one process from start to finish

Do not start by reviewing all the data the company has. Pick a process to automate and trace a few real cases.

If you are automating order handling, note: where the order is created, where payment information comes from, who changes the status, when the invoice is issued and which system passes on the tracking number. Mark the places where an employee copies data, corrects it by hand or contacts someone else to confirm it is right.

In the end you should have a short list of the data the process needs to run. For example: order number, customer ID, payment status, invoice details, products and their prices, delivery address and tracking number. Only then check which systems hold that data.

This approach quickly separates an important problem from a cosmetic one. An inconsistent name for a marketing campaign can wait. A missing unambiguous order number, on which issuing the invoice depends, has to be solved before the automation goes live.

Step 2. Define what each important field means

Two systems may both have a “status” field, but it need not mean the same thing. One system treats an order as completed once the parcel is packed, the other only after delivery. When an automation relies on the status, such a difference affects customer communication and reports.

Prepare a simple dictionary of the most important fields. For each, write the name, the meaning, example values and the system where it is created. It is also worth adding the format of dates, currencies, phone numbers or product codes, if this information is to be passed between tools.

It does not have to be extensive documentation. For the first process, a single table is often enough:

Such a table is useful both to the business team and to the person building the integration. It answers the question of what rules the automation should follow.

Step 3. Choose the authoritative source for each piece of information

“Source of truth” sounds technical, but it means a simple rule: when two systems show different values, you need to know which one to trust.

It will not always be one system for the whole company. The store may be the right source for order data, the accounting program for issued invoices, and the warehouse system for product availability. What matters is naming the source separately for the information used in a given process.

Also decide who can change it, and where. If an employee corrects an address in the CRM, but the automation soon overwrites it with an older version from the store, synchronisation alone will not solve the problem. You need an update rule and a way to check when a given value was changed.

Organising data therefore does not have to mean migrating everything to one program. Clear rules for exchanging information between existing systems are often enough.

Step 4. Check for duplicates and identifiers

The same person may appear in the store with a private address, in the CRM with a business address and in the accounting program under the company name. Before you merge these records, decide when they actually describe the same customer.

An email address alone is not always enough: one mailbox may be used by several people in a company. A phone number alone is not certain either. For orders, it is best to rely on a unique order identifier and to store the links to the identifiers in the other systems. For customers, the matching rule may need several data points and a manual check in ambiguous cases.

Do not merge records automatically just because their names look similar. A wrong merge of two customers can be harder to fix than leaving a duplicate to be reviewed.

While you are at it, it is worth standardising the formats of the data used for matching: postcodes, phone numbers, dates or product codes. That way the system will not treat two ways of writing the same value as two different pieces of data.

Step 5. Define what is an error and what is an acceptable gap

An empty field does not always mean the same thing. A missing flat number can be fine. A missing email address may make it impossible to send a confirmation. A missing VAT number on a private person’s order is different from a missing VAT number on a company invoice.

For the fields the automation uses, settle three things: when the field is required, which values are acceptable and what should happen when a value is incorrect. Possible actions include holding the order, retrying the data fetch, asking the customer to fill in the information or passing the case to an employee.

It is worth measuring how often problems occur in real data. For example, check what percentage of orders have complete invoice data, how many products have no unambiguous identifier and how many times the order status differs between systems. There is no single threshold of “good data” for every company: the requirements should depend on the consequences of a mistake and on how quickly it can be detected.

Step 6. Plan a path for exceptions

Not every case has to go through the automation to the end. A well-designed automation knows when to stop and hand the decision to a person.

If two systems give different order amounts, the safer solution may be to hold the invoice and notify the right person. The employee should see which order the problem concerns, what values were found and what needs checking. Once the data is corrected, the process can continue without someone performing all the remaining steps by hand.

At the same time, separate data errors from failures of the connection to a system. A missing VAT number calls for a different response than the accounting program being briefly unavailable. The second problem may justify an automatic retry; retrying the request will not fix the first.

Step 7. Record what the automation did and why

When a customer reports a problem, the company should be able to trace the path of their case. A practical record is enough: the order or ticket ID, the time the step ran, the source of the information, the decision taken and the result. The aim is not to collect unlimited technical logs, but to be able to answer the question: “Why did the system send this message or issue this invoice?”

Such a record also helps you develop the process. If every week a dozen cases go to manual review for the same reason, you know what to fix in the form, the integration or the validation rules.

How do you launch automation without tidying up the whole company at once?

Start with one workflow and a limited number of real cases. Before launch, check orders or tickets that are typical, incomplete, duplicated and contain contradictory information. Make sure it is clear who handles exceptions and how to switch off or halt the process if the results turn out to be wrong.

For the first few weeks, watch mainly:

• how many cases the automation handled correctly without human help;
• how many cases went to manual verification, and why;
• whether duplicates or conflicting statuses appear;
• how long a case takes to handle before and after the change.

After you change a form, an integration or the way data is stored, go back to a few test cases. A small change to a field the automation uses can have more impact than a major change to how an application looks.

Frequently asked questions

Do you have to move all your data to one system before automating?

No. If you know which system is responsible for a given piece of information, how the systems identify the same records and what to do when data conflicts, you can automate a process that uses several tools.

Which data should you organise first?

The data the automation uses to make decisions: identifiers, statuses, dates, amounts and the information needed to carry out the next step. Fields the process does not use can often be left for later.

What should you do if data is incomplete?

First check where the gaps come from. Sometimes it is enough to fix the form, add a check before the order is saved or change the moment the information is fetched. If the gap is acceptable, describe it in the rule. If it makes it unsafe to perform an action, send the case to be completed.

How do you know whether the problem lies in the data or in the integration?

Examine a specific failed case. If the systems were working, but the value passed on was empty, in the wrong format or contradicted another field, the problem most likely lies in the data or in the rules for processing it. If correct data did not arrive because a service was unavailable, the connection failed or an API limit was hit, the problem lies in the integration. A record of how the case ran lets you tell the two apart.

Summary

Good data preparation does not mean correcting every entry in every system. It means organising the information a specific process depends on: its meaning, source, identifiers and rules of control. Just as important is deciding what the automation will do when data turns out to be incomplete or contradictory.

If you plan to connect a store, a CRM, accounting or other tools, start by tracing one real process. At Internet Stars, we help define its rules, prepare the integrations and implement automation that your team can keep under control after launch.