OpenAI has a huge conflict of interest in calling for AI regulation: Zerodha CTO Kailash Nadh

I think it is not even possible. This is not like building nuclear weapons, where there are humongous physical limitations and you can detect if a country is sourcing the material to build it by its physical footprint. But, someone with a bunch of servers in the basement and enough money can build AI. No state can know whether somebody else is building AI or not.

You can write a small, simple GPT in a few hundred lines of code at home. The only limitations today may be the availability of data and server farms. And this is not a big deal for a state actor. I don't think signing a global pact saying that certain countries will not develop AI would be meaningful. The good actors might stop there, but the bad actors and rogue actors will happily continue building stuff.

Also, there is no superintelligence just yet. It might happen tomorrow or it might never happen. We don’t know. But this, coming from OpenAI, is a bit wonky.

Do you think OpenAI is going overboard in asking for regulation?

OpenAI has been saying that there has to be heavy regulation and even suggested licensing schemes for AI development. I think most voices out there are critical of OpenAI’s stance because of its conflict of interest.

Now that they have built it, any regulation might have to grandfather them and give them an unfair advantage and a moat. And, newcomers will be stifled. There is a huge conflict of interest in them calling for really strict regulation for something that they just created.

ALSO READ: Part 1 | 'Most tech jobs in IT services can be automated with generative AI'

What does it mean when it is said that GPT models have a black box problem? People built it with an outcome in mind which has been achieved. Why then do they not understand how it works?

Computer programs typically work by writing precise code to achieve a certain objective. Every word in the code has a specific purpose. You can also introduce some randomness into the code and then pinpoint in the outcome that a certain behaviour was observed because of that randomness. So, even that is explainable.

However, when you are writing the software for AI/ML, you are writing just a framework or a shell of code. There are no lines of code that you write in a machine learning system that says that when somebody says ‘hi’ or ‘how are you’, what should be the response. Instead, you take vast amounts of data from the internet — trillions of words of textual data — and feed it into the shell of code. It then uses that data to form connections between the words in abstract mathematical terms.

You don't tell the LLM that a cat always appears in the context of a dog because they're both pets. But, such relationships automatically emerge because of the vast amount of text knowledge where cats and dogs typically tend to appear together. Those relationships start getting abstracted to a higher level inside the system automatically.

For example, if you tell ChatGPT that Gandhiji was using a smartphone and he clicked a photo and uploaded it to social media, it would say that it's not possible because smartphones and social media did not exist during his lifetime. Now, anachronism is a concept humans understand. How did GPT detect it? You can’t really peek into the memory and figure it out. This inability to pinpoint why a certain input led to a specific output gives rise to the black box problem.

In medical science, it is said that no one really knows how general anesthesia works at a molecular level. Some physicists might say wave-particle duality is not understood at a fundamental level, and yet atomic physics has progressed a great deal. Can’t the black box problem of AI be similarly put on the backburner?

Even though we may not understand anesthesia at a molecular level, we understand it enough to use it safely on a large scale. The other example is a secret of the universe, whether we understand or not. But, drawing a comparison between these and AI would be a false equivalence. While anesthesia only puts people to sleep, AI would be increasingly used to do many things — by individuals and corporations. So, it needs a more nuanced approach.

When you start introducing black box technologies into decision-making across millions and millions of small and large decisions in society, it is really dangerous if we don’t understand how they fundamentally work. A trivial example is that AI is starting to be used by corporations that get thousands of resumes. If we don’t understand how it works, we can’t know how it is filtering the resumes — if there are certain biases in the AI model.

In such cases, won’t companies have human reviewers to check biases?

It really depends on incentives. In an ideal world, people would use technologies very carefully. But, I don't think it's practical, because that's not how society or organisations really behave. They would use it to their advantage to maximise efficiency, minimise costs… the same old incentives. If an organisation can do with one reviewer instead of 10, they have a reason, legally, to do that.

A couple of months back, people like Elon Musk, Steve Wozniak, Max Tegmark and others wrote a joint letter that called for a 6-month pause on further AI development. What’s your take?

The question ideally should be ‘should humans develop AI or not?’ Should we hold back human progress, like we held back cloning of humans, right? Because there were very clear, perceived problems with that. It was also far easier to define the problems and the ethics of human cloning, compared to AI. It will push humanity forward massively and at the same time, pull it backward massively in other aspects.

But the whole thing about pausing AI development for six months is a joke. It makes zero sense whatsoever. Why six months? Why not four months? Why not 12 months? Either you pause indefinitely, saying that the day when humanity understands all of this stuff we will resume AI. Or you don't pause at all.

AI is writing movies, music and novels, and we are believing the output. There seems to be an emotional element. It's not just AI mastering purely logical stuff like chess anymore. Is that in some way indicative of some kind of AI sentience?

Not really. This is an extremely contentious topic. It’s a 2,000-year-old discussion around free will, sentience, cognition, what it means to be human etc. There are some people who argue that LLMs are showing signs of sentience. But I don't think the large majority of people, philosophers, AI researchers are claiming sentience here. Even the definition of sentience is very unclear. Not everyone agrees on one singular definition of sentience.

I think it really goes back to Searle’s Chinese room experiment.

AI has been more powerful in chess than top grandmasters since the 90s when IBM’s Deep Blue beat Gary Kasparov. You can even make two machines play with each other. Yet, the fan following of top chess players has not been affected. If at all, it may have increased. Can this be taken as an indication that musicians, painters, writers won’t lose out to AI?

When a human makes an amazing move in chess, it’s amazing because the human did it with human limitations. If AI makes an amazing chess move, nobody cares because it can do anything. But, chess is bounded as human brilliance for a limited set of moves within a limited time period is lauded. If an AI can generate a 1,000 page novel, yes, there are lessons we can take away from the chess example, but I think it's going to be different.

If one AI can generate unlimited content forever, then maybe that form of content does lose value, right? I have a sense that when the proliferation happens, where a bulk of the content is being generated by AI, a niche will emerge for human-created or handcrafted content... Scarcity also has a place in the idea of creativity. When JK Rowling writes something, people value it because she wrote it, because of her history and her body of work.

Copyright infringement lawsuits are being filed against generative AI companies. Some artists don’t want their work to be used to train AI models, whereas human artists have always been drawing ‘inspiration’ from others. What is your take?

I think this is also a philosophical problem that needs to be solved before it can be resolved legally. The definitions of originality, creativity, and learning must be re-examined in the age of AI. If I listen to 10,000 music tracks before composing a new song, of course I will be influenced by what I have consumed. Only if there is a significant overlap with any of those 10,000 tracks would there be grounds for copyright infringement.

Any creative work always is influenced by a million other things that we've learned consciously and subconsciously. Whether the output of an AI model that has been influenced by millions of images is original or not, is to my mind more of a philosophically ambiguous problem first.

The Indian tradition has always been different from the Western tradition in terms of authorship. We don’t know the names of the original creators of ragas, vedas or upanishads. People are thought to have always adapted earlier versions through the generations. Can the Indian tradition provide an argument for open-sourcing knowledge in the age of AI?

We can draw that analogy, but I don't know if there is a meaningful connection to be made. We probably didn't really have the concept of individual attribution because of the oral tradition. Writing is very new to our civilisation, and when you transmit things orally over thousands of years, information gets lost.

When AI consumes a lot of data, learns from it and produces something new, it is not exactly in the spirit of collectivism or open source. It's not like an AI has naturally, like a human does, gone out and learned things. These are all millions of images, text snippets, etc. that corporations have scraped from the internet. And, one of the biggest issues right now is that people are saying that this was taken without their consent.

Is there a way people can prevent their data from being scraped off the internet to train AI models?

Not only AI models, but search engines like Google have scraped all information on the internet without asking anyone. It's exactly the same thing, right? Just that Google shows snippets of what it has scraped, while AI models are creating new things out of it. That's the only difference. But, in fact, Google has sucked up more data than any of these AI models.

There are certain websites out there who say that they don't want to be indexed by Google. So, there's a technical mechanism called no-index for that. Some large corporations might introduce a technical measure like no-index, but this is only for the law-abiding corporations. Anybody else can just copy paste and scrape whatever is on the internet. While Google honours the no-index thing, some random corporation or the millions of people who will do this in their own spare time may not honour it.

If something is available on the internet, with or without copyright, it might just be subsumed by an AI model. It’s not possible to prevent it.

ALSO READ: Part 1 | 'Most tech jobs in IT services can be automated with generative AI'