Feature Article: 3 Metadata Tasks that GenAI Can Automate
As part of our series on Metadata, DAM and AI, Acquia’s Jake Athey explores the various ways in which DAM users can automate common metadata workflows using generative artificial intelligence. The article takes a look at the findings of a recent study designed to test the automation of three common tasks that traditionally require a significant amount of human time and effort: keyword tagging for product imagery, product descriptions, and generating alternative text (alt-text) used by screen readers to improve accessibility for visually impaired users.
“In these use cases, DAM admins care most about two qualities: accuracy and precision. Accuracy is about whether AI identified the content and context of an image correctly. In the JDMM study, we evaluated that based on a one-line product description. Precision is about whether keywords were factual, relevant, and plausible as search terms. Depending on how you prompt GenAI, the results can vary a lot.” [Read More]
Jake continues by describing the prompt-engineering template that was used to guide the AI model, based on six key aspects which were programmatically modified in order to compare accuracy and precision when run against a set of 60 images. The results of the exercise were somewhat mixed, with the most accurate prompts still providing a sufficiently high number of erroneous, misleading or irrelevant descriptions to render them unusable for both product descriptions and alt-text purposes.
“The top-performing prompt, run against a generic version of GPT-4, wasn’t perfect. Its accuracy rate of 88% means that 12% of the product descriptions it generated were dead wrong or too flawed to be used for anything, like alt-text. Its precision score of 91% means that 9% of the keywords it tagged were untrue, misleading, or unhelpful to DAM users.” [Read More]
The article concludes by noting that although GenAI isn’t perfect, with careful crafting and adequate human oversight it can still be more efficient than doing everything manually.
You can read the full article at the link below.
https://digitalassetmanagementnews.org/features/3-metadata-tasks-that-genai-can-automate/
Share this Article:
Accurate GenAi requires a lot of upfront human oversight for testing and analyzing. Once a rich prompt is created that results in the accuracy percentage you are looking for, the rest of the automated workflow should work well. It is interesting that there is a “human-like” aspect to getting the results you want from LLMs, it is noted that “LLMs seem to get sidetracked if you don’t remind them of their mission often enough.” In the provided example the goal of the project is reiterated three times, is there such a thing as too many reminders or too many prompts in the template? How much can you prompt before the parameters are too narrow? I assume these are items that are tested in the first phase of working with GenAi in order to create the perfect (or as close to perfect as one can get) automated workflow.