Configurable Workflow Systems
This special feature was contributed by Remko Noteboom, Co-Founder and CTO, Southpaw Technology.
Overview
Today, every organization and company in the world is struggling to keep up with the proliferation of digital assets. As a result, digital workflow technologies have become mission critical technologies anywhere large quantities of digital content and data are being created.
Within various industries, there are a number of solutions – both desktop applications and enterprise-wide solutions – that are being used to manage various aspects of the digital data lifecycle. Typical solutions include:
- Digital Asset or Media Asset Management systems (DAM, PDAM, PAM, MAM)
- Project Management solutions (PM)
- Business Process Management systems (BPM)
- Software Configuration Management solutions (SCM)
- Content Management Systems (CMS).
Despite the fact that there is much talk about workflow management across every industry, there has been no comprehensive solution offered by traditional vendors that allow companies to support any part of, or all of, the digital workflow.
This article lays out a vision for workflow management, and how configurable workflow systems could become more and more critical to businesses across a range of vertical markets and solutions.
The Digital Assembly Line
Over the last two decades, there has been an explosion in digital content creation. Much of that explosion began in the media and entertainment industry, but is now spilling into the enterprise space, where big data has become a core challenge.
Much of this growth in data creation has resulted from the exponentially increasing pace of underlying technology. Hardware infrastructures, coupled with increasingly powerful software tools, produce an enormous amount of data. The demands this data places on teams and technology is staggering: 1000’s of CPU farms, 100’s of Terabytes of data into the Petabyte range, and GB+ networks.
One of the more common means of managing all of his digital data in enterprise environments is digital asset management (DAM). However, the vast majority of DAM solutions are concerned with either the archiving of finished digital assets or the delivery of finished digital assets to an end user. Very few are concerned with how digital assets move through an enterprise, how they connect to specific departments, people, tasks and deadlines, and where they go when they reach their end of life. In other words, DAM’s ability to manage workflow is limited.
In the early 1900’s, Henry Ford developed techniques that would revolutionize the manufacturing world. It is striking that most digital processes have not migrated to using these proven techniques of mass production well known in the manufacturing world. Digital content is still largely produced using the same methods of the craftsman of old.
Complexity is one of the main reasons vendors have failed to tame the digital assembly line. In today’s economy, the production line has to be much more flexible in order to produce, move, manage and store data and assets. Unlike the physical world, mass production of identical digital assets is trivial. However, in a digital pipeline, employees must produce a large number of similar but ultimately different assets. This variance is often one that requires a human to judge, criticize, comment on, and possibly, alter. This is where a digital process differs from a physical one.
The scalability of a digital workflow or “assembly line” lies in its flexibility. It is not any solution’s place to dictate the workflow; it should only be flexible enough to support, secure and speed up the workflow; intelligent enough to package and deliver an asset from one step of the process to another, and versatile enough to allow people to work the way they want in creating the asset, regardless of the type of content. It also requires a foundation of careful design coupled with an intricate understanding of the process.
Deliverables and Digital Workflows
In order to track the process by which someone creates digital data, it must be broken down into a pipeline, which consists of a flow chart of individual processes connected together. Taking a step back, one can see that this precisely describes the assembly processes invented and developed by Henry Ford, but instead of cars, it is being applied to a digital process. Specialization of labor merely translates to the workflow within the process. Efficiency is gained by designing a set of processes that best utilizes the talents and specializations of the people and resources available without hindering them under the weight of an overbearing system.
To translate this to digital workflow, the same methodology of designing a digital assembly line can be performed. Effective work process relies on defining exact deliverables from one process to another. Pipelines break down when each receivable must be “translated” or worse “cleaned up” at each stage.
It is therefore critical for a process down the pipeline to have the authority to reject a delivered package that is not packaged in a way defined by the design of the overall pipeline. This “reject” pushes back the deliverable and forces the previous processes to re-package it. In other words, each deliverable should be packaged properly in order to make the next set of processes as efficient as possible. It becomes even more efficient when the system itself is able to package the deliverable, removing this duty from the creators.
Workflow management systems offer this type of digital assembly line. The system needs to be configurable enough to support, speed and track the progress of any process while securely storing every deliverable in an organized fashion. When a process “delivers” to processes downstream, it checks them into the workflow system. When a process “receives” a deliverable from a previous process, it goes to the workflow system for the deliverable. In other words, the workflow system is intricately intertwined with the entire pipeline, and in essence, becomes the backbone for parts of, or all of, the digital assembly line.
The Search Engine
Fundamental to any workflow system is the ability to search for assets. However, no Google-style, tag-based indexing algorithm will ever be sophisticated enough to construct an engine, a building or any complex digital asset. In order to achieve this, a different organizational structure for assets must be used, one in which everything has its place and is organized around a strict set a rules based upon the design of the workflow. This requires the assets to be organized not according to tags and indexing, but according to its “place”.
Traditional DAM systems require a huge effort of tagging each individual asset. This is a very painstaking and involved process because each asset must be categorized after completion of the asset. With a workflow oriented system, the resulting tags or metadata associated with an asset are a natural byproduct of the creation process and very little time is spent on creating tags – it is a natural extension to the creation process.
Configurability is Critical
When an entity (company or organization or large project etc.) starts out, the entity often defines a workflow for the creation of data. As the entity grows, the workflows become more numerous, the tools to assist in those workflows proliferate, and the amount of digital data grows. Ultimately, entities are forced to develop larger and larger infrastructures to support the workflows. That infrastructure may encompass new and legacy tools and technologies, such as databases.
At the same time, no two entities are alike. What widgets they make might differ. How they make their widgets might differ. The combination of these disparate processes and technologies requires workflow solutions to be infinitely configurable.
They must integrate well with databases or they won’t be able to access or move digital files and data. They must offer some method for customizing the user experience, or they won’t be adopted across the enterprise.
Ideally this infinitely configurable solution must be able to reach across enterprise level database schema, possess a plugin architecture with customizable HTML5 themes that would offer a great user experience.
A Case for Open Source
The upfront costs of adapting a comprehensive workflow solution to incorporate numerous internal and external workflows can be large, making it difficult for some enterprises to justify.
Open source systems do not have these staggering initial costs. The code is easily and rapidly accessible (no legal or business affairs effort up front) so internal programmers can start working immediately in their own sandboxes. This allows for teams to present a better use case to justify budget for additional services or support. Rather than paying to have systems integrated into an existing infrastructure, open source systems enable companies and organisations to design a system that works for them, utilizes their legacy investments and all at a pace that works for them.
Conclusion
The pace of digital content creation and the digitization of business processes will not be slowing down. Most stakeholders would agree that the pace will continue to accelerate and continue to infiltrate all organisations. As the pace accelerates, so too will the need for configurable workflow systems that efficiently make, manage, move and store digital files.
Organisations and companies are beginning to realize that they cannot ignore the inefficiencies of the digital workflows within their operation. Inefficiencies cost money. Meanwhile, these processes and these assets are essential to the success of the company.
Just like the assembly lines of old (Henry Ford era), new age digital workflows must be there to ensure the scalability and configurability of the digital process. Organisations and companies that ignore the “industrialization” of mass digital production do so at their own peril and they won’t be able to ignore it forever.
Share this Article: