English 中文(简体)
Pys Park CI/CD 管道
原标题:Pyspark CI/CD Pipeline

经常是一米学习花园,以开展电子职业教育。 在此,我问,如何为PyS火药铺设一条标准管道。 在Kubernetes使用PyS花园,有空气流,我不知道如何制定标准版本和工艺。

我对此没有想法。 此前,我曾使用Talend用于ETL,因此Talend本身用于管道、工艺品和所有物品。

如何进行版本和工艺品的编写,如果PyS引发文字故障,如何回归旧版本?

问题回答

利用SemVer打上你的PyS花图。 http://stackoverflow.com/questions/55044480/tagging-docker-image-semver"post。

Later, if you want to rollback, just update the image tag in the Airflow operator that submit your PySpark job to Kubernetes.





相关问题
Is Web Service suitable for ETL purpose?

My company is considering using web service as mean of ETL process. However I don t think web service fit into this purpose, for several reasons: 1. web service could possibly consume a lot of memory ...

What is the best file parsing solution for converting files?

I am looking for the best solution for custom file parsing for our enterprise import routines. I want to basically change one file format into a standard file format and have one routine that imports ...

SSIS Package failing with New structure of Flatfile

SSIS package is just importing from txt file to sql database. when we made the package were using old file and its executing fine.the old source file got (10 columns) the new source file got 15 ...

Is Pentaho ETL and Data Analyzer good choice?

I was looking for ETL tool and on google found lot about Pentaho Kettle. I also need a Data Analyzer to run on Star Schema so that business user can play around and generate any kind of report or ...

ETL Tools and Build Tools

I have familiarities with software automated build tools ( such as Automated Build Studio). Now I am looking at ETL tools. The one thing crosses my mind is that, I can do anything I can do in ETL ...

热门标签