How, and why, do enterprises adopt streaming data applications in the cloud?
To better understand this trend and the underlying software stack needed to do it right, Lightbend has partnered with The New Stack for its second charity survey on the trends around cloud-native, real-time streaming data applications.
A total of 804 IT professionals provided details about their prototypes, projects, and production applications that use stream processing at their organizations–and created over $800 in charitable funds towards Girls Who Code.
Get your own copy of this dive into industry trends to learn more about various topics affecting how we build, deploy, and manage streaming data applications in the cloud...
The use of stream processing for AI/ML applications increased four-fold in two years. Those already using streaming for AI/ML expect this trend to continue with even broader use in the coming year.
Developer experience, familiarity with tools, and technical complexity are barriers to adoption. Concern about scalability, latency and other technical challenges increases as the number of workloads utilizing stream processing rises.
Persisting data in a microservices architecture becomes less of an issue as users gain more experience with containers, microservices and modern databases. Architects see a future where stream processing and microservices are deployed in the same container-based infrastructure stack.
While Kafka is sufficient for ETL and messaging, it faces robust vendor competition among streaming platforms for advanced use cases such as IoT pipelines and recommendation engines.
