Will your AI models take Code of Conduct training and an audit?

Microsoft AI has released a draft Code of Conduct for its AI models and opened it for public feedback. I read the document and sent my inputs through the consultation form. It is worth reading: microsoft.ai/code-of-conduct.

What the draft gets right

I like that they have put intended model behaviour in writing and opened it for comment before it is final. The chain of command is easy to follow, and the constraints on the model are clear. It was also good to see the document admit openly that it does not describe how the models behave today.

Legible reasoning traces, agreed stopping conditions, minimum privilege, and treating tool and web content as having no authority are all good to see. Those are exactly the human control points we deal with when building agents.

What is missing: how anyone will know

My main concern is that the document says a lot about what the models should do and very little about how anyone will know they are doing it. Four things I would like to see addressed.

1. Where each rule is enforced. It is not clear which rules are meant to be achieved through training, which through runtime controls such as verifiers, classifiers and monitoring, which are left to operators under contract, and which would need an outside party to check. A simple table mapping each constraint to where it is enforced would make the document far more useful to operators and regulators.
2. Verification before release. The document mentions red-teaming and pre-deployment review but gives no release criteria, no minimum results a model must meet, and no role for independent testers. External partners appear only as a future plan. I would like to see a commitment to publish evaluation results against the intended behaviours for each model release, and an explanation of how this work lines up with existing assurance and governance standards around the world. Otherwise each company ends up with its own code and there is no common way to compare them.
3. Checks for operators. Operators take on responsibility for the model, but they get no way to check that a configured or fine-tuned deployment still meets the absolute constraints. Most enterprise deployments will involve fine-tuning, smaller models or custom system prompts. It would help to know whether the constraints are expected to hold after fine-tuning, and whether operators will get evaluation sets they can run against their own use cases.
4. A way to report problems. There is no route for users or third parties to report a model breaking the Code, or to find out what happened afterwards. And since the models are not yet trained on this document, a short note on the status of each section would help readers separate current behaviour from future intent.

A process on paper is not a process in practice

I hope the next version takes the same care over assurance. Otherwise it will remain a document. Anyone who has rolled out a new process or methodology in a large organisation knows this: writing the process down is the easy part. Putting it into practice is a different story, even with all the alignment and collaboration an enterprise can muster.

While reading, I also found a lot of related work happening across the industry. The momentum still needs to build, and I think governments and regulators will need to play a big role in lining it up.