Announcement: New Design for docs.informatica.com!
We have updated the look and feel of docs.informatica.com. To load the latest design, press CTRL-F5 to clear the pages you previously cached on our site, or simply restart your browser.
Data Engineering Streaming uses third-party distributions to connect to a Spark engine on a Hadoop cluster or to a Databricks Spark engine on a Databricks cluster.
In a Hadoop environment, Data Engineering Streaming pushes job processing to the Spark engine. It uses YARN to manage the resources on a Spark cluster.
In a Databricks environment, Data Engineering Streaming pushes job processing to the Databricks cluster, and the Databricks Spark engine runs the job.