This is a huge question for me. Why did alchemy persist, as an idea, from 300 B.C. to about 1750 A.D.? Over 2000 years. And it never worked. Why did it last?

Support for Alchemy

There were a few things that could be observed by practicing alchemists to support the ideas of transmutation.

"Transmutations" Chemical reactions typically start with one or two substances which transform into a new substance with entirely new properties. It looks like transmutation when observed. One of the thrills of inorganic lab is the reactions where the colors change: combine burgundy red with clear and you get black, which crystallizes as orange. Magical! It's mostly a property of transition metals, like iron, and the old alchemists worked in iron pans for the high-temperature processes, and iron is thought to have dissolved into the reaction mixture, so they would have seen the colors of iron.

Inorganic salts of iron.

"Copper Mine Effluents" Late in alchemy it was observed by miners that if you put a piece of iron into the water running out of a copper mine that soon the iron was coated with copper. It looked like an alchemical transmutation. We know it to be redox chemistry, specifically a single-replacement electrochemical reaction. But it looks to all the world like a transmutation, at least on the surface.

Even with those few supports, transmutation never worked. Surely the alchemists would have given up in the first hundred years of trying, and it would have never survived past the time of Christ.

Niche Theory

Biologists and botanists observe that some life exists only in certain places and in certain conditions, and cannot live outside that place. This is a biological niche. Particularly frail species can only exist in a niche, and should the niche change, the species is lost.

Social ideas operate on the same principle. Some ideas are impractical, and will not survive a pragmatic world. Business is pragmatic, and rarely supports a niche (unless that niche is willing to spend money for nothing in return other than support the niche; i.e. social media). For example, people who believe that society owes them money for no reason cannot survive in society until they find a place where people are given money for no reason. That place, or situation, or government body, becomes their niche. They are fragile and cannot live outside it.

Academia is a famous niche for liberal ideas, ideas which cannot survive the marketplace because they are unrealistic and few pragmatic citizens support them, citizens operating on principles of practicality for their family's benefit.

Sometimes an idealistic idea propagates into society. But that ideal while disturbing to the practical-minded while it remains avant-garde, will never survive the marketplace. They never have. Idealistic ideas need a niche in which to live. There exists a rich history of temporary large-scale niches: pet rocks (gone), ALAR (gone), vaccine-autism links (small niche), the red menace (gone), witchcraft (gone), civil defense (gone, but for the prepper niche), etc. Some niches become entirely societal, because we managed to make them practical: automobiles, computers, photography, air travel, pens, electronic music, digital photography, digital music, movies; the list is long.

A Social Niche Protects Impractical Ideas

Universities are famous for harboring impracticality, because the work there is to teach ideas, not to live by selling the consequences of those ideas, as business does. This is why liberals flock to the University; at their core liberals are idealists, and idealism needs a niche. 

Alchemy, who's fathers are Plato and Aristotle, were idealists, as Plato and Aristotle were idealists. They both formed a world-view from principles, and forming beliefs from principles is the recipe for idealism. Alchemy is protected in a niche formed by idealism, and propagated by other alchemists, particularly their writings.

This blog series, I see now, is a record of that idealism, of that niche, of how these ideas were protected from practicality.

How did alchemists ignore the fact that they could not perform a transmutation? It is documented here in their writings.

How did they entice the next generation of young people to hold to their ideals over practicality? It is also documented here.

How did alchemists withstand the assault of the practical over the ideal? That is documented here.

Idealism

Idealism has many, many forms, all bad. The crusades were nothing but misery and woe, except if you had the great luck to die early. They were driven by utter idealism. Communism is idealistic, and it has only created misery for the people (though the leaders mostly seem to do well). Most of governments biggest problems are the consequence of policies and programs idealistically set up to alleviate misery, but consequently generated vastly more misery ("let's make sure no one is without money in this temporary economic setback by establishing a permanent program giving away so much money that we create a sub-population who will not work because we will give them money, to the point we need to go further into debt to pay the interest on our debt until it fails when we need to tax away everyone's money").

I think I can categorically state that idealism is bad in every regard. Idealism turns people into fodder. People become impediments to the ideal when they do not agree with the ideals or fail to live up to the perfection demanded by the ideal, impediments that must be removed so the ideals can flourish. Idealism trumps people.

The thing I like most about those I know who are not idealists is that they care about people. This is an honest, practical care for others, because that's all we have for help. This is the sort of care I like most. Idealism makes me fit into the mold shaped by the ideal, and that just isn't me. As a personal observation, I've spent most of my life masking as an autistic person, and I know what it's like to spend most of your time with others being someone you are not. It's misery, and I don't like it at all.

Avoid the niches of the idealistic. They will dehumanize you. Jesus was all about this: spend your time with the homeless, those in jail, the laborers, the wanderers, and avoid those who live in a niche, like the academic and rich. Don't give money to idealistic snake charmers; they are really about personal enrichment and only use the ideal to pry open your wallet. And don't believe any fore-tellers of the future: they are idealists who have proven to be correct (even their conception of current reality is warped by their ideals), and will invariably invoke an ideal to cover their poor prognostication.

In a general sense, the idealist probably prefers deduction, finding specifics from generalist ideas; the realist prefers induction, forming a general rule from many observations, or inference, using past and present observations to form an up-to-date general rule describing all observations so far.

Here is a little joke I saw online today:

So what of University? As I said, it collects the idealists. I'm not one of them. I simply love teaching. Teaching scratches an important autistic itch I have. And it does so very well. There are others in the Chemistry department who are practical, and I love them all. They came to UVU from industry and learned their practicality there. But others at the University, and a few in our department, are idealistic, and largely annoying. Biology has tons of them. Go to a small school, one with an agriculture program, or a plumbing program or some other practical trade. Good work helps keep the idealist away. Find a school with a small administration; the idealists set up huge administrations to maintain their niche. But above all, avoid any place or anyone who takes themselves seriously; they are pretending to an ideal, and they are bad for you.

And avoid cancel culture: those are idealists trying to maintain their niche. In general it's best to support the cancelled; they are being practical, unless they support a different ideal, in which case run away and keep your head down! No good comes from those fights. Diogenes was a realist of the best sort. Find out who he was.

Good luck out there, and keep it real.