You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Encountered following error when I run LSTM model with orca pyspark tf2 estimator:
Traceback (most recent call last):
File "lstm_pollution.py", line 33, in<module>
steps_per_epoch= df.count() // batch_size)
File "/home/cengguang/anaconda3/envs/bigdl2.1/lib/python3.7/site-packages/bigdl/orca/learn/tf2/pyspark_estimator.py", line 199, in fit
lambda iter: transform_func(iter, init_params, params)).collect()
File "/home/cengguang/anaconda3/envs/bigdl2.1/lib/python3.7/site-packages/pyspark/rdd.py", line 816, in collect
sock_info = self.ctx._jvm.PythonRDD.collectAndServe(self._jrdd.rdd())
File "/home/cengguang/anaconda3/envs/bigdl2.1/lib/python3.7/site-packages/py4j/java_gateway.py", line 1257, in __call__
answer, self.gateway_client, self.target_id, self.name)
File "/home/cengguang/anaconda3/envs/bigdl2.1/lib/python3.7/site-packages/pyspark/sql/utils.py", line 63, in deco
return f(*a, **kw)
File "/home/cengguang/anaconda3/envs/bigdl2.1/lib/python3.7/site-packages/py4j/protocol.py", line 328, in get_return_value
format(target_id, ".", name), value)
py4j.protocol.Py4JJavaError: An error occurred while calling z:org.apache.spark.api.python.PythonRDD.collectAndServe.
: org.apache.spark.SparkException: Job aborted due to stage failure: Could not recover from a failed barrier ResultStage. Most recent failure reason: Stage failed because barrier task ResultTask(12, 0) finished unsuccessfully.
org.apache.spark.api.python.PythonException: Traceback (most recent call last):
File "/home/cengguang/anaconda3/envs/bigdl2.1/lib/python3.7/site-packages/pyspark/python/lib/pyspark.zip/pyspark/worker.py", line 377, in main
process()
File "/home/cengguang/anaconda3/envs/bigdl2.1/lib/python3.7/site-packages/pyspark/python/lib/pyspark.zip/pyspark/worker.py", line 372, in process
serializer.dump_stream(func(split_index, iterator), outfile)
File "/home/cengguang/anaconda3/envs/bigdl2.1/lib/python3.7/site-packages/pyspark/rdd.py", line 2465, in func
return f(iterator)
File "/home/cengguang/anaconda3/envs/bigdl2.1/lib/python3.7/site-packages/bigdl/orca/learn/tf2/pyspark_estimator.py", line 199, in<lambda>
lambda iter: transform_func(iter, init_params, params)).collect()
File "/home/cengguang/anaconda3/envs/bigdl2.1/lib/python3.7/site-packages/bigdl/orca/learn/tf2/pyspark_estimator.py", line 196, in transform_func
return SparkRunner(**init_param).step(**param)
File "/home/cengguang/anaconda3/envs/bigdl2.1/lib/python3.7/site-packages/bigdl/orca/learn/tf2/spark_runner.py", line 349, in step
"optimizer_weights": model.optimizer.get_weights()
AttributeError: 'Adam' object has no attribute 'get_weights'
I use tensorflow 2.11 and estimator.fit() API with argument model_dir specified. Seems that the program use deprecated API to get weights from optimizer. Please check if we need to update the usage.
The text was updated successfully, but these errors were encountered:
lalalapotter
changed the title
Update usage of Depracate API (get_weights) in TensorFlow 2 Optimizer.
Update usage of Deprecated API (get_weights) in TensorFlow 2 Optimizer.
Jan 19, 2023
lalalapotter
changed the title
Update usage of Deprecated API (get_weights) in TensorFlow 2 Optimizer.
Update usage of deprecated API (get_weights) in TensorFlow 2 Optimizer.
Jan 19, 2023
Encountered following error when I run LSTM model with orca pyspark tf2 estimator:
I use tensorflow 2.11 and
estimator.fit()
API with argumentmodel_dir
specified. Seems that the program use deprecated API to get weights from optimizer. Please check if we need to update the usage.The text was updated successfully, but these errors were encountered: