Can technology become too complex that the civilization that created it can no longer understand it? How can humans delay or prevent such an outcome?
After entering the industrial era, under the guidance of the theory, humans tried to simplify the "individual component model", based on which to construct "more complex technology", such as steam engines, internal combustion engines.
to promote the development of civilization.
More than two centuries, the "complex technology system" has emerging, there is machinery equipment that we cannot live without and information systems that make global civilizations condense.
But technology is getting more complicated:
"Technology complexity is getting higher" is reflected in: When we verify and design a product,
After the complexity has developed to this stage, the theory involved in technological advancement become too complicated.
In the era of "Capital is king", the high complexity caused by technology is likely to stifle its development.
The "inter-relatedness" and "inter-embedding" among the technology have increased their complexity, but they greatly improved human capabilities at the same time.
Therefore, human beings have the motivation to improve it continuously.
After the industrial revolution, capital became active, and lead "Technological Complexity" expanded exponentially.
This is strange and contradictory: Capital promoted the "upgrading of technology complexity," but today it hates "the high complexity of tech.
Technical complexity, like "muscles on our body":
I’ll tell you about something that exists right now that no group of the smartest people in the world can figure out.
Neural network algorithms are a type of artificial intelligence used every day.
Google uses advanced algorithms to give you top-notch search results.
YouTube recommends videos to you using algorithms.
Self-driving cars utilize insanely complex neural networks to recognize if a human is in their path or not, so they can stop in time.
Quora even uses them to suggest the 10 most interesting questions to you in your daily Digest.
Neural networks are all over, and they’re important.
However, nobody knows how they work.
Imagine a complex neural network that a programmer feeds a billion pictures of humans into, then a billion pictures of different cars into it (with the programmer explaining whether each picture is an image of a human or a car).
If this example sounds familiar, it probably is.
Has Google recently asked you to identify pictures of road signs, cars, or streets for its reCAPTCHA, to make sure you’re not a bot? If so, you’ve become the programmer telling the algorithm the answer to the image.
Coincidentally, Google is developing its own self-driving car right now, and you’ve become free labor for them to help train their algorithms.
After feeding it billions of pictures, the algorithm gets pretty good at predicting whether any additional image is a car or a human.
But, we have no idea what happens to the data after it enters the “black box”.
This is a representation of a neural network (it’s called a neural network because it vaguely resembles the brain’s neuron structure, another piece of biological ‘technology’ we have no idea how it works).
The circles represent “nodes” and the lines follow the data as it is inputted, stretched, contorted, bended, and finally outputted to the user.
While we can figure out what single nodes do, and we can figure out what small clusters do, we have absolutely no idea how the data goes in one end and comes out the other end.
Like, we just don’t.
Now, there’s a catch.
If you want high interpretability (i.
, you MUST know how the data is manipulated from start to finish) you will have to stick to a relatively less complex, less reliable neural network.
It might correctly identify a car vs.
human 60% of the time, but at least you’ll know how it works.
On the other hand, you can have a super complex, super accurate (>99%) neural network, but you must sacrifice the knowledge of how your data is being handled.
MIT professor Dimitris Bertsimas gave an interesting but lengthy on this if you care to take a look.
So, to answer your question: It’s already happened.
We already have technology that we don’t understand, yet we still use it a ton in almost every single industry on earth (that should make you a little worried).
And it’s only being used more and more.
Hurrah for progress, I guess?
Interestingly enough, this is something that we deal with every single day.
Science cannot explain humans’ relationship with money.
This is one of the first coins ever minted:
It’s made of electrum— a mixture of gold and silver.
It was minted about 2,500 years ago.
It’s physical properties were unimportant to those people who used it— only its properties were important.
The tradition of the use of money continued.
We know that the money itself is not valuable, because humans will accept warehouse receipts as substitutes for physical money.
They look like this:
Nowadays, they look like bits in a computer— which don’t really look like anything at all.
The Physical Sciences Cannot Tell Us Why
Interestingly enough, we have no scientific procedure which can tell us what money is in the sense of the term which would be meaningful to us.
Money is a peculiarly human phenomenon whose properties exist beyond physical measurements.
This to say that the properties of money cannot be understood in purely physical terms— in those terms, the coin I showed you above is just a crudely stamped amalgam of gold and silver.
I can call someone to make you another just like it, but we would both likely believe that to not be “worthwhile.
For most things, if we wanted more of them we would physically create more of them.
With money, its physical reproduction does not serve the purpose for which we seem to use it— our uses for money are interpersonal, and involve its exchange rather than its creation.
Money seems to be one of those phenomena which are only important to humans.
My Own Answer
Another example would be the concept of writing— any written work’s value is purely metaphysical and cannot be proven scientifically.
A written work is of no interest to us in terms of its purely physical properties.
There’s no physical, scientific distinction between the works of Shakespeare, Oscar Wilde, and myself.
However, the works of Shakespeare and Wilde are worth far more money than anything I’ve written.
No one has mentioned this, and since it seems to be exactly related to the question, I thought I’d mention it.
It might not come up in computer science algorithm courses—at least none I’ve had so far and I’ve taken all but advanced algorithms, a course called “automata”—but it did come up in a philosophy course called “minds, brains, and computers” when discussing connectionist networks and artificial intelligence.
Bonini's Paradox, named after business professor , explains the difficulty in constructing or that fully capture the workings of (such as the human ).
In modern discourse, the paradox was articulated by John M.
Dutton and William H.
"As a model of a complex system becomes more complete, it becomes less understandable.
Alternatively, as a model grows more realistic, it also becomes just as difficult to understand as the real-world processes it represents".
This paradox may be used by researchers to explain why complete models of the human brain and thinking processes have not been created and will undoubtedly remain difficult for years to come.
This same paradox was observed earlier from a quote by philosopher-poet , "Ce qui est simple est toujours faux.
Ce qui ne l’est pas est inutilisable".
("A simple statement is bound to be untrue.
One that is not simple cannot be utilized.
Also, the same topic has been discussed by in his classic essay "The Strategy of Model Building in Population Biology", in stating that complex models have 'too many parameters to measure, leading to analytically insoluble equations that would exceed the capacity of our computers, but the results would have no meaning for us even if they could be solved.
This has, in fact, already happened.
Take a look at your pencil, for example.
A fairly simple device.
Do you know and understand every step of its manufacture, without having to “google” it?
Now take a look at your smart phone, or even the computer you are using now.
Do you understand everything about it? How it works, how it was produced? How the OS works? How the silicon chips were manufactured? What about the keyboard? Do you understand about the “n-key rollover” and how that’s achieved?
Or for that matter, the Internet, which has become an integral part of all of our lives.
Do you fully understand how it works? All the many protocols, how the routers and switches make decisions on how to route the packets — do you even understand what the packets are?
And I can do the same with the car you drive every day, the food you buy — the logistics behind getting it from the farm to the factory to your local supermarket?
Technology has already exceeded the point where no one individual can hope to understand it all.
And why would you want to prevent this outcome anyway? Shall we go back to the horse and buggy, the telegraph, perhaps? Maybe we could have the old Model-T Fords, where a single person could possibly understand everything about them.
Most people don’t even comprehend something as simple as electricity; what happens when you flick on the light switch.
Nor do they understand how florescent and LED lamps work.
They may understand incandescent bulbs, but even that may be pushing it.
And the so-called “neural nets” that are in great use today in everything from voice recognition to autonomous vehicles? No one understands how they work in detail.
Just how to train them and test them.
And let’s not even talk about the Large Hadron Collider, the most complex machine man has ever built.
The level of physics and engineering that went into it boggles even my mind.
It reminds me of those mysterious machines in Forbidden Planet.
The complexity of technology will only continue to increase.
There is no stopping it.
All we can hope to do is to make responsible choices with how we use it.
Can technology become too complex that the civilization that created it can no longer understand it?
A true, though unbelievable story…
In the late 1980s, I wrote a custom system for a company that did everything for the user but make their coffee.
It had been chugging along for months with no problems, until one day I got a phone call from an operator complaining that the system wasn’t working.
Over the phone I went through the usual remote diagnostics, and the operator described everything to me that the system should have been doing upto an input point.
At that point, the User said they couldn’t do any input.
There was over a one hour commute between offices, but with no other choices, I got in my car and drove across town to the customer’s site.
When I got there, the screen was blank.
On interviewing the operator, she admitted to me that what she had described to me over the phone is what she had observed every other morning when she flicked the on-switch on the box.
This prompted me to look at the back of the machine, where I discovered the box wasn’t plugged into the power supply.
How can humans delay or prevent such an outcome?
Euthanize the idiots, or at least sterialise them so they cannot replicate, and put them somewhere where they caan’t do any damage.
It’s already happened, a few times over.
The best example of this would be our current financial system.
Modern capitalism isn’t a naturally occurring phenomenon, it’s something that was developed slowly over time.
However, even though it’s purely a human creation, there’s still a lot about economics that we don’t understand, it’s extremely difficult to model, and historically we’ve had massive financial crashes due to a lack of understanding.
I suppose a more modern example would be artificial neural networks.
For those unfamiliar, a neural network is a basically a computer algorithm that can train itself to make predictions without human input.
That’s a gross oversimplification, but you get the idea.
We’ve developed these models that can make amazing predictions, and we know techniques for improving the performance of neural networks.
But we don’t have a full working theory of how they work (e.
no one can definitively say when a technique will help or hurt the model).
The reason technology can outpace human understanding is that even very simple systems can have very complex interactions.
And beyond that, we can develop a technology through trial and error without fully understanding the theory behind why that technology works.
For example, early humans figured out farming without any knowledge of genetics or chemistry.
And we developed batteries long before we had a theory of electromagnetic forces.
It’s also worth noting that it’s not necessarily a bad thing for technology to outpace human understanding.
Yes, there’s always a danger for unintended consequences when you’re using a tool that you don’t understand.
But the fact that we can develop tools that we don’t understand has pushed inquiry forward into exciting new realms.
Yes! This has already happened, in the evolution of Microsoft.
They just keep putting band-aids on things.
There is no human being in the world who really knows what the code is doing.
Whenever software is created by a team of people, it potentially falls into the category you’re suggesting.
The creators had their own subsets and subspecialties – the software finally came to the point where it was finished, and published and sold … and WHAT!? … something needs to be patched or updated?
Instead of taking that enormous hunk of code back, to fix the problem, since it’s already distributed, we’ll just hand out these superficial band-aids to patch the problem … and if it happens again, we’ll patch it again!
Sometimes, people who came up with great coding ideas moved on, and work somewhere else.
Years go by, people die; the teams change.
Microsoft became a big balloon of bandaids.
Nobody really knows what’s in there! Years and years of patches has resulted in a real mess.
Original question: Can technology become too complex that the civilization that created it can no longer understand it? How can humans delay or prevent such an outcome?
We also get better at the technology of explaining technology.
A relevant example from my own career.
In 1960, Two PhD computer scientists named Backus and Naur struggled to explain the syntax of Algol.
They wrote a whole book on how to do it.
When I was in college, their Backus Naur Form was provided essentially without explanation to undergraduates.
We’d figured out how to explain it (plus we realized it wasn’t that hard to explain).
Another relevant example: When I was in elementary school, the teachers just presented a simplified form of the material one time.
It was good enough for most students.
When my chilldren were in elementary school, the teachers provided a whole bunch of tools to organize their learning.
They provided rubrics of what was expected in an assignment.
I don’t know if my children learned more than I did, but they certainly had the opportunity to.
There’s nothing like having a little history with a technology or mechanism to make it easier to explain.
Good question, but that is not possible.
This is the life cycle of technology:
unknown -> science fiction -> complex technology -> common knowledge
Complex science, when used, taught and learned on regular basis, becomes “second nature”.
We then move on to something more complex, which in turn becomes redundant and second nature eventually.
This cycle of technology evolution continues.
Type writers of yesterday are too simple of a machine for today’s kids.
Back then, “machine learning” was science fiction, today it is complex science, tomorrow it will be second nature and we would eventually move to something more complex that is unknown today.
Discovering the fact that earth is round when most thought that it is flat was really complex science back then.
Today kids can explain you most scientific phenomenons that were discovered in past few hundred years, which were really complex piece of science back that.
Most of us understand how steam engines and car engines work today.
Imagine the days of horse based carriages.
Though today scram-jets and cryogenic engines are complex machineries that propel a space craft, very soon they will be “common knowledge” and we would have discovered something more complex by then, which is probably science fiction today.
There are already many forms of technology so complex that our current civilization does not understand it fully.
One very pertinent example is the financial market.
No one fully understands how it works completely, which is why, even with the collective efforts of millions of people working together, we cannot avoid market crashes, or perfectly predict when they will happen, and our governments and corporations cannot react optimally to minimize the damage.
Another, much simpler example, is the bicycle.
The full physics of how bicycles work is not completely understood.
Part of the development cycle for new bicycles requires simple trial and error, because we cannot predict how certain changes in certain parts will work.
There are other examples like the bicycle, where we invented a technology through tinkering and trial and error until getting something that works, without understanding exactly how it works, and only figuring that part out much later.
And in some cases, like with the bicycle, we still haven’t figured the thing fully out, even as we continue to use it.
As we build new technology, we also build new technology to support it (metatechnology).
This question assumes a linear growth of knowledge.
This is never the case.
If we were trying to build computers the way Greeks were building a clepsydre, your assumption would be correct.
There is no way, Platon could have managed today’s technology, with ancient Greeks concepts and tools.
Nowadays, computer chips, are built by machines themselves built upon powerful technologies.
We have extremely powerful softwares expanding the natural abilities of our brain to sort, organize and search information (ask google).
The technology we build has vastly expanded our life expectancy, soon it will go beyond mere tools, and allow us to speed up the growth of our brains, and our natural abilities.
Some are pointing out how our species has already started adapting to the new world we created in less than a century.
So hell no, we won’t be left behind by our own work, provided we manage to stay out of world wars, and focus on developing our economies and peacful wealth creation.
The prospects are brights, very brights, even for the poorests.
Hunger and poverty has been steadily regressing over the last century.
A couple billion extra well fed brains will add to our collective ability to create and master new technology, feeding forward the virtous cycle, we’re in.
We used electricity for years without understanding it.
“Technology” usually goes in stages with us:
First, we use the technology without realizing we’re using it.
Think quantum mechanics.
We’re made of it, we had no idea we were made of it, we still “use” it all the time (i.
we breathe) and have no idea how it works.
Then, we “discover” the technology and begin to understand small things about it, enough to build a machine with it.
For years we use the machine, which depends on this technology, without ever really understanding what’s going on inside the machine.
Electricity was like that, as was magnetism.
Finally after years and years of research we come to a nearly full understanding of the “technology” and we figure out how to manipulate it and use it more fully.
This happens with all technology, and will continue to happen.
For example, in the Neural Network case mentioned in another answer, we’re currently in stage 2.
Stage 1 was when we used neural networks in our brain.
Stage 3 will be when we fully understand how the neural networks operate and we start building them smaller and faster and eventually make a “positronic brain” or whatever the realization of that idea is.
We don’t have to look too far into the future, or at cutting edge technologies.
It would be an understatement to say that we don’t understand some devices that use fluid flow as a part of the engineering process – especially aerodynamic devices.
Aerodynamics and fluid flow are so complex that we cannot predict the exact behaviour of aircraft, under all circumstances.
We only have guesses of how well they work in most operational conditions.
We routinely see phenomena such as lift hysteresis in aircraft wing designthat are not directly explainable by the linear systems we build using the laws of aerodynamics and fluid mechanics.
This is rarely good enough for explaining real world behaviour (which can mostly only be simulated from initial conditions).
The related phenomenon of non-linear dynamics routinely puts us in the domain of uncertain system behaviour.
We can even build very simple devices such as double pendulums, which we can barely describe the physics of.
These are devices which behave predictably for small perturbations, but when the perturbations involve a lot of non-linearity, the behaviour becomes chaotic (which is a physics term taken to mean sensitive dependence on initial conditions).
Overall, this kind of inability to explain the systems we build is a very common phenomenon – much more common than we think!
Nobody understands the whole technology stack.
Even back in the day, archers didn't necessarily know how to fletch arrows.
Technology relies on division of labor.
Everybody learns part of the system.
People are finding uses for newer "machine learning" systems where the solution to the problem kind of evolves and while we understand how to evolve a solution we don't understand how the solution itself works.
We understand each of the pieces, just not how they work together to achieve the result.
Eventually we might have a machine system we don't actually understand, which then produces another machine system that it doesn't actually understand.
At that point no human will understand the solution, or how to produce it.
It doesn't really matter if we understand it as long as it works.
If we don't understand it, and it's contrary to our interests, then it's analogous to a natural event.
We've been dealing with those since forever.
This point has long since past.
Nobody knows how everything works, and nobody has for a long long time.
Setting aside the issue you raise, which is a valid one, of complexity, there is the sheer unrelenting volume of things to know.
Thousands of fields with thousands of sub fields and details details details.
But I’d like to speak to a different reason, and that is secrecy.
We have trade secrets designed to protect businesses from having their products or work copied by others.
We have government secrets designed to prevent terrorists, or other governments, from gaining certain capabilities.
You cannot know how to do everything, because you cannot be a member of every competing company, you cannot work in every department of every government in the world at once.
And this isn’t news.
In the past, we had guilds who held knowledge of techniques secret, and governments who held knowledge of techniques secret.
To this day we do not know how Wootz steel was produced (modern “damascus” is similar but not identical, and we don’t know why), we do not know what Greek Fire consisted of… these secrets were not just held against others of their day, but have been lost and to the best of our knowledge not rediscovered.
Unless someone knows but is keeping it a secret.
First off, we live in a universe that is so complex that even with 2000 years of science we do still not understand it.
That is without technology!
I work with the internet, the actual network, not web servers or other applications.
It is all based on protokolls that we expect every single component to adhere to.
I do not care about how they make that happen as long it is correct, fast and reasonably priced.
I have understanding of most of the components involved.
I teach in the subject.
But just this part of the phenomenon that is the internet is way waaaay to complex to “understand”.
We have to break it down to concepts that we as humans can handle.
Just as the dude who optimizes the silicone designs does not really understand facebook.
Facebook developers do not understand the work he does.
But without it facebook would not exist.
We have to create conventions and standards and work with them.
Even society needs social conventions to work effectively or we would not get anything done and get most thing wrong.
These concepts and conventions are paramount to civilization and no one is an expert outside of their field.
I could only see that happening if either:
a) There is a global catastrophe that wipes out most of the world’s population and infrastructure so that, over the course of a few generations, people no longer have access to the tools and knowledge that were used to create it.
b) An A.
comes into being that is smarter than the smartest human, at which point its intellect will begin to expand exponentially and inevitably create things beyond a human mind’s comprehension.
That’s not to say there aren’t every day situations in the world today where a technology is not understood or supported because of limitations in that specific situation.
I used to work for a company in the late 90s with an extremely old, text-based CRM system.
They’d lost all their in-house developers and were unable to back fill because it was written in an old language that was no longer common (don’t recall what it was).
When the Y2K scare came, they ended up having to fly in some out-of-state consultants in order to “Y2K-proof” the code.
They pored over years of code and (after much trial-and-error) got it done.
Not to say this example was a pinnacle of human achievement, but my basic belief is that any technology created by a human can be reverse engineered by a human given enough time and resources.
Several of the answers here seem unjustifiably ironic in tone.
Adam makes a telling point about the financial markets, though, and we all see that technology itself is having a clearly difficult time controlling it’s impact on the environment.
So this is a serious issue.
Can specific technologies, namely computer systems run amok and gain consciousness a la Kurzweil’s thinking? I don’t know.
But the existential effects of technology are enormously impactful.
Here in Tokyo, and all over the world, we are all glued to little screens, or big ones, and do we understand what this is doing to us? The young forget what pre-online life was like.
Personally, I miss it sometimes.
I just read a book (!) Singh’s Fermat’s Last Theorem, and the experience is so different from random surfing through Salon and Huffington and the NYT.
So although it’s hard to imagine computers really adopting identities and coalescing to take over the world, I can visualize a kind of second-level human transformation that is already taking place as a result of the constant intrusion of devices and the ethernet.
It may not be “bad,” but is may be making us, somehow, into “aliens.
First, define “civilization that created it can no longer understand it”.
Since humans do not have a communal mind, this criteria is ill-defined at best.
There are already many complex and dynamic systems that any one individual cannot completely understand: modern computer operating systems, modern military aircraft or ships, scientific machines, such as the or the , and so on.
There are even realtively simple chaotic systems that cannot be predicted, so from one viewpoint, they cannot befully understood.
Second, why would we want to “delay or prevent such an outcome”? Rather, we want to protect against possible negative consequences of this complexity.
We need to develop the ethical frameworks (example ), international standards and law, and associated controlling and protection systems to ensure optimal use and safety of these technologies.
It is already that way for many people who struggle to understand why things they expect to work just can’t be done.
The amount of Quora requests for answers that are related to technology and science that people have misconceptions about are staggering.
People see, hear, and read things that are purely fictitious and they ask why it hasn't it been done.
The answer is often that although we should think we can do something amazing with our current technology that amazing something has been ruled out by lengthy and concise studies.
The possibility is so far fetched, the probability is incalculable, and our ability lacks the power of any control of certain aspects that for the foreseeable future (or actually past that mark) it can be said it is ruled out as a legitimate possibility.
I do not appraisal’s or value diamond and do not buy, sell or .
I think this is quite a likely scenario given the rate at which technology advances (I think I read somewhere that we are coming up with new ideas 10 times more than 10 years ago, but don’t quote me on that statistic).
But yes, the growth is exponential.
I don’t believe the solution lies so much in delaying or preventing this from happening.
When technology does become this advanced, it would be crucial to make sure it is used for benign purposes.
The main issue would be controlling it.
Here is a brilliant video from Recode where billionaire CEO Elon Musk discusses his fears for AI.
My thoughts resonate with his.
Here’s something else to fry your mind: supposedly it is a logical argument that the universe we know and understand (or don’t really understand) is a computer simulation created by beings like us to preserve their consciousness.
Then, technology has already surpassed us and we can’t figure out how it works :)
I don’t think complexity is the problem.
From what I can tell, an awful lot of civilizations have declined or fallen resulting in lower standards of living among the population living in the area.
And while many of these civilizations have had the occasional difficult to understand technology (e.
, Greek fire, Roman cement), and vast supply chains, the reality is there was very little there which wasn’t relatively straightforward.
There was very little about Rome that a reasonably intelligent Hun or Vandal couldn’t understand were he or she to wander Rome’s ruins.
Rebuilding Rome – the city and the concept – could have, in theory, been done very quickly.
Building out the same infrastructure somewhere else was also, in theory, doable.
It wasn’t lack of understanding that prevented it.
Similarly, I think most San Francisco residents wish their city was as clean and as orderly as Tokyo.
And most Cariocas wish Rio de Janeiro was as clean and as orderly as San Francisco.
And most Harare residents wish they lived in a place as clean and as orderly as Rio.
It isn’t complexity that prevents all of those wishes from coming true.
For whatever reason the people who live in places make choices that lead to the outcomes we observe.
If all high technology disappeared tomorrow Tokyo would still be cleaner and more orderly than Harare a decade later.
In fact, that is probably part of the reason why Tokyo has more complexity than Harare.