Why It’s Important
Embedding generative AI into platforms can significantly support enterprises efficiently by retrieving and organising vast amounts of information to rapidly uncover insights that might be missed by human analysis. However, its limitations and the potential for consequential impacts in critical areas require a more cautious and monitored approach to its use. ServiceNow acknowledges that its tool still has its limitations and solely relying on AI for decision-making, especially in critical areas like healthcare or legal, can be risky.
The company also reiterates that it collects and uses the inputs, outputs, and edits to outputs of Now Assist to improve its models and AI products. Enterprises must consider opting out of future data collection to protect their sensitive information.
Who’s Impacted
- AI developers
- HR service delivery teams
- Customer service teams
- IT Teams
What’s Next?
- Continuously test AI outputs against real-world scenarios, particularly for use cases in critical sectors like healthcare or finance.
- Validate the AI’s performance for specific use cases to ensure it meets the required standards.
- Set up processes where critical decisions are reviewed by human experts, especially in high-stakes scenarios.