Trusting Data

We have all heard customer support people tell us "that is what the computer says" even when we knew that the computer was wrong. Computer data is often trusted when it shouldn't be and not trusted when it should be. Computers and data need to cross a "trust boundary" and things go wrong at that boundary. We face liars and out dated information all the time. We need to have a good process for letting data inside that trust boundary.

Spammers often take advantage of our trust by sending messages supposedly from someone else who we trust. We get emails that look like they are from the bank. We get messages supposedly from friends who have been abducted or are in a foreign jail.

The same is true of business data. Anyone who has purchased a mailing list knows that a large percentage of the addresses are wrong. No matter how "verified" the data is, a large percentage is wrong.

Part of the reason for that incorrect data is that no list can keep up with reality. People move. People switch jobs. I know one person whose wife insisted that they change their phone number every time she got a phone call where nobody seemed to be on the other end. That meant that they were changing their phone number at least once a month.

The Census Bureau reports that the national average move rate is about 12 percent per year. Young people move much faster at about 30 percent per year and renters as a group move at about 24 percent per year. No list can keep up with that pace.

Stephen Covey's book on the Speed of Trust shows that being able to trust speeds up processes and organizations. That is why we trust data that is within our trust boundary. However, we need to clearly identify where the trust boundary is and what the process is for letting data within that boundary.

Computer systems can generate insane amounts of data but not all of it is to be trusted. When building a web site, all incoming data has to be treated as untrustworthy. We have no way of knowing where that data came from or if malicious data is buried deep within the flow. The best sites validate every bit of data to make sure that that bit fits what is expected at that moment in that context and disallows any bit that does not fit.

Organizations have multiple ways of enforcing the trust boundary. Financial institutions insist that systems be certified before interconnecting. This process validates not just the computer protocols but also the physical security of those systems. Other organizations use techniques such as JSON encoding, XML documents, and other industry specific standards to ensure that the data being transferred can be trusted.

Trust allows us to operate much faster and to change quickly. Both are valued in business. However, building the environment where trust can happen is hard work. Our trust boundaries are constantly being challenged and tested. We do well to spend time insuring that they hold.

Costs of Speedups

In the Large Hadron Collider, the second run was designed to run at a rate of one burst every 25 nanoseconds. To put that in perspective, in 25 nanoseconds, light travels only about 8 feet. There was tremendous costs and effort to be able to get some information when things are happening that fast.

In business, we are all being pushed to speed up processes. Amazon and Walmart are competing to see who can deliver the fastest. Nearly every business is facing the same pressure to speed things up.

It costs a lot to speed things up. Both Amazon and Walmart have significant distribution networks to get the goods from production to the consumer. It takes investment in both information systems and distribution to get delivery down to less than 24 hours.

To put things in perspective, in 1919, an Army convoy took 56 days to travel from Washington DC to California. Practically all roads from Illinois through Nevada were unpaved and the convoy sent scouts on ahead to find the path. We have come a long way in a short time.

In all of these, it took commitment, planning, design, and hard work to get the speed up to happen. That we can travel at high speed from coast to coast took billions of tax dollars as well as many years. We can not just wish our way to faster processes.

The same is true with speeding up any venture. It takes commitment, planning, design, and hard work to speed up any human venture.

We see the same thing in software development. Often a prospect wants some code written quickly but also to run fast. Those are mutually exclusive. Fast running code is written slowly.

Risky World

Yara, a shipping company in Norway, has commissioned a ship that they plan to convert to fully autonomous operation - needing no crew. This was reported in the same week as a report of pirates electronically hijacking a ship, remotely taking over its navigation system, and intending to drive it to where its cargo could be taken. No crew would mean no way to tell if the ship had been hijacked.