Question

这是我下面的法典,我正在发现这一错误“火花园。产出栏目已经存在”,即检查其他员额,但我不相信需要做些什么才能在这里帮助任何人。

from pyspark.sql import SparkSession
from pyspark.ml.feature import StringIndexer, VectorAssembler
from pyspark.ml.classification import LinearSVC
from pyspark.ml import Pipeline
from pyspark.ml.evaluation import MulticlassClassificationEvaluator

# Create a SparkSession
spark = SparkSession.builder.appName("").getOrCreate()

# Load your TSV file into a DataFrame
data = spark.read.csv("sleep.tsv", sep= 	 , header=True, inferSchema=True)

input_cols = ["V0", "V1", "V2", "V3", "V4", "V5", "V6", "V7", "V8", "V9", "V10", "V11", "V12"]

# Concatenate input columns into a single column named "features"
assembler = VectorAssembler(inputCols=input_cols, outputCol= "features")
data_assembled = assembler.transform(data)

# Renamed target column to label
data_assembled = data_assembled.withColumnRenamed("target", "label")

### Split data into training and testing sets
(trainingData, testData) = new_data.randomSplit([0.8, 0.2], seed=16)

### Fit the pipeline to training data
model = pipeline.fit(trainingData)

Answer 1

之所以出现错误,是因为“名称”一栏是在转变过程中制造的不止一次。为了解决这个问题,你应确保“名称”一栏仅设立一次。

你可以更新以下法典:

页: 1

   data_assembled = assembler.transform(data)

2- 更新VerctorAssembler输出一栏的名称,使之具有独特性,例如,“组装”:

 assembler = VectorAssembler(inputCols=input_cols, outputCol="assembled_features")
data_assembled = assembler.transform(data)

3- Update your subsequent code to reflect this change:

 //Renamed target column to label

data_assembled = data_assembled.withColumnRenamed("target", "label")

# Define your pipeline including the VectorAssembler step
pipeline = Pipeline(stages=[assembler, ...])  # Add your other stages

# Split data into training and testing sets
(trainingData, testData) = data_assembled.randomSplit([0.8, 0.2], seed=16)

# Fit the pipeline to training data
model = pipeline.fit(trainingData)

友情链接