AI regulations that are too weak or aim at wrong companies can unintentionally make AI systems less safe, as per the new study published in the Proceedings of the National Academy of Sciences (PNAS). The researchers from Cornell University and Carnegie Mellon University argue that effective AI regulation should focus on companies making the foundation models rather than only the businesses deploying AI in products and services.
The study suggests that when regulations primarily target downstream companies such as firms using AI in healthcare, customer support or e-commerce, developers of general purpose AI models may reduce their own investments in safety measures. Instead, they could rely on application providers to handle safety risks, creating gaps in oversight.
Using economic modelling and game theory, the researchers examined how the different regulatory approaches influence the behaviour of AI companies. They found that model developers are more likely to scale back efforts such as third party safety audits if they believe downstream businesses will bear the responsibility of ensuring safe deployment.
The researchers describe this as a free riding problem, where AI developers shift the burden of safety onto companies building applications on top of their models. According to the study, this may ultimately result in AI systems that are less secure than those developed without such regulations.
The study comes when the entire world is debating on how AI should be governed. In the US, one group says that minimal regulation is needed to maintain innovation and compete globally while others are pushing for stronger safeguards to address concerns from misinformation and mental health impacts to job displacement.
The researchers, on the other hand, argue that stricter, well-designed regulation does not necessarily come at the expense of innovation. As per their model, needing meaningful safety investments from both AI developers and companies deploying AI can improve overall system safety.