[Solved] Problems with TensorFlow 1.0 and Windows

[drfknoble]

Hi there,

First, I wanted to say thanks for sharing! I'm working through the code to help with my own segmentation project and having something to work from is a big help.

Second, I came across a few issues (minor really) that I've figured out and wanted to share:

• TensorFlow 1.0 replaced tf.pack() with tf.stack().
• In TensorFlow 1.0, variables should be initialised using tf.global_variables_initializer()
• In Windows, the os.path.splittext() should use "\ \", rather than '/'. Otherwise, the program can't find any files to pickle (and the MITSceneParsing.pickle file is empty), which in turn means 0 records are found and the feed dict instruction doesn't work.

Like I said, pretty minor stuff, but I wanted to post in case anyone else had any issues.

Best regards,

Frazer

P.S. If you get an out of memory error, it's likely because you're trying to work with 20,000 images, which might be a bit too much. I deleted some of the training images and it worked.

[jiao0805]

Hi

Can this project be successfully run on windows ?
My tensorflow is 1.0 gpu version on windows.

[drfknoble]

Hi Jiao0805,

Absolutely, I've trained this neural network on a Windows 8 and a Windows 10 Machine, using both TensorFlow CPU and GPU versions. I'd recommend you use Anaconda to manage your Python environments and use PIP to install the version you're after.

Here's a sample output from my application, which detects lanes on a road. For reference, I used the KITTI data set (see http://www.cvlibs.net/datasets/kitti/).

For my Windows 8 machine, I had to run TensorFlow on my CPU, as my GPU doesn't have enough memory (you'll need > 4 GB for this repo' to run). Last, but not least, give it at least 30, 000 training steps to get reasonable outputs; 10, 000, as a minimum, should start to give tentative results.

Best regards,

Frazer

[jiao0805]

Hi FKNoble,
Thanks a lot for your reply!
I use tensorflow 1.0 gpu on windows 7 , and I saw the issues you shared above.
I did all of them but I still got one error as follows:

Traceback (most recent call last):
File "FCN.py", line 221, in
tf.app.run()
File "C:\Users\SEELE\AppData\Local\Programs\Python\Python35\lib\site-packages
tensorflow\python\platform\app.py", line 44, in run
_sys.exit(main(_sys.argv[:1] + flags_passthrough))
File "FCN.py", line 152, in main
loss = tf.reduce_mean((tf.nn.sparse_softmax_cross_entropy_with_logits(logits
,tf.squeeze(annotation, squeeze_dims=[3]),name="entropy")))
File "C:\Users\SEELE\AppData\Local\Programs\Python\Python35\lib\site-packages
tensorflow\python\ops\nn_ops.py", line 1684, in sparse_softmax_cross_entropy_wit
h_logits
labels, logits)
File "C:\Users\SEELE\AppData\Local\Programs\Python\Python35\lib\site-packages
tensorflow\python\ops\nn_ops.py", line 1533, in _ensure_xent_args
"named arguments (labels=..., logits=..., ...)" % name)
ValueError: Only call sparse_softmax_cross_entropy_with_logits with named argu
ments (labels=..., logits=..., ...)

It seems that there is something wrong with my cost function.
Did you meet this error on win8?
Could you please kindly tell me how can I solve it?

[drfknoble]

I had something similar and what I think it means is that the cross entropy function needs the input parameters to be explicitly labeled, i.e. labels=labels, logits=logits, etc.

Here's the function that I have, which works:

loss = tf.reduce_mean((tf.nn.sparse_softmax_cross_entropy_with_logits(
logits=logits,
labels=tf.squeeze(annotation, squeeze_dims=[3]),
name="entropy")))

Let me know if it fixes your problem.

Best regards,

Frazer

[jiao0805]

Hi FKNoble,
I fixed my problem and I found that the cross entropy function actually needs four parameters.

Please have a look on this:

def _ensure_xent_args(name, sentinel, labels, logits):
#Make sure that all arguments were passed as named arguments.
if sentinel is not None:
raise ValueError("Only call %s with "
"named arguments (labels=..., logits=..., ...)" % name)

I found them in the declaration of tf.nn.sparse_softmax_cross_entropy_with_logits()
I tried your code and my problem fixed.
I also tried this:

None,
tf.squeeze(annotation, squeeze_dims=[3]),
logits,
"entropy")))
And it also worked.

And as for the os.path.splittext()
To split URL, use '/' , to split file path, use'\'
otherwise program can not found the file.
see here:

SceneParsing_folder = os.path.splitext(DATA_URL.split("/")[-1])[0]
filename = os.path.splitext(f.split("\\")[-1])[0] 

And I finally run into a error as follows:

Setting up Saver...
Step: 0, Train_loss:316.906
2017-03-14 12:25:00.698600 ---> Validation_loss: 424.702
E c:\tf_jenkins\home\workspace\release-win\device\gpu\os\windows\tensorflow\stream_executor\cuda\cuda_event.cc:49] Error polling for event status: failed to query event: CUDA_ERROR_UNKNOWN
F c:\tf_jenkins\home\workspace\release-win\device\gpu\os\windows\tensorflow\core\common_runtime\gpu\gpu_event_mgr.cc:198] Unexpected Event status: 1
E c:\tf_jenkins\home\workspace\release-win\device\gpu\os\windows\tensorflow\stream_executor\cuda\cuda_driver.cc:1034] failed to alloc 536870912 bytes on host: CUDA_ERROR_UNKNOWN
W c:\tf_jenkins\home\workspace\release-win\device\gpu\os\windows\tensorflow\core\common_runtime\gpu\pool_allocator.h:195] could not allocate pinned host memory of size: 536870912
...
...
...
E c:\tf_jenkins\home\workspace\release-win\device\gpu\os\windows\tensorflow\stream_executor\cuda\cuda_driver.cc:1034] failed to alloc 536870912 bytes on host: CUDA_ERROR_UNKNOWN
W c:\tf_jenkins\home\workspace\release-win\device\gpu\os\windows\tensorflow\core\common_runtime\gpu\pool_allocator.h:195] could not allocate pinned host memory of size: 536870912

Is that mean that I do not have enough memory?
My graphics card has 6GB memory.

Looking for your reply!

Best regards,
Jiao

[shekkizh]

Thanks @FKNoble for helping out.

@jiao0805 Your last error does seem like an issue with insufficient gpu memory. try reducing batch size or image size. Also it would be helpful if you post different issues in a separate thread - it will be useful for people in the future to make note of. Thanks.

[BowieHsu]

@FKNoble thanks for your sharing, but after i modified my code as you said, when i try to training the network, i still got loss:Nan results, my running environment is windows 7 + TITAN X + python3.5 + tensorflow 1.01, training set is ADEChallengeData2016 with num_of_classes = 151

[chankeh]

I used the version 0.12rc, but it occured like this：

Traceback (most recent call last):
Found pickle file!
File "E:/Python/FCN/FCN.py", line 224, in
tf.app.run()
File "D:\Program Files\Anaconda3\lib\site-packages\tensorflow\python\platform\app.py", line 43, in run
sys.exit(main(sys.argv[:1] + flags_passthrough))
File "E:/Python/FCN/FCN.py", line 168, in main
train_records, valid_records = scene_parsing.read_dataset(FLAGS.data_dir)
File "E:\Python\FCN\read_MITSceneParsingData.py", line 30, in read_dataset
training_records = result['training']
TypeError: 'NoneType' object is not subscriptable

how can i fixed this problem？
thanks.

[CLegendBuptSun]

Sorry to bother you.
When I test the random batch(visualization),I get the answers of nearly black prediction mask，I read your code top to the bottom，but didnot get the reason。
Then，I changed the batchsize=16 and learning rate=1e-4， MAX_ITERATION =50000，finally，I got
something but nearly close to black to...
I want to know WHY.
thank you for your reply.

[drenego]

I think you can try with batchsize=1 and learning rate=1e-5 max_iteration>=100000, then you will get an acceptable model, at least i did this.

[zhshining]

Hi FKNoble
Sorry to trouble you .
I ran FCN.py in my windows by cpu Tensorflow, but there were some questions :
Traceback (most recent call last):
File "D:/DL/data/FCN/FCN.tensorflow-master/FCN.py", line 223, in
tf.app.run()
File "D:\DL\Tensorflow\anconda\lib\site-packages\tensorflow\python\platform\app.py", line 48, in run
_sys.exit(main(_sys.argv[:1] + flags_passthrough))
File "D:/DL/data/FCN/FCN.tensorflow-master/FCN.py", line 194, in main
sess.run(train_op, feed_dict=feed_dict)
File "D:\DL\Tensorflow\anconda\lib\site-packages\tensorflow\python\client\session.py", line 889, in run
run_metadata_ptr)
File "D:\DL\Tensorflow\anconda\lib\site-packages\tensorflow\python\client\session.py", line 1096, in _run
% (np_val.shape, subfeed_t.name, str(subfeed_t.get_shape())))
ValueError: Cannot feed value of shape (0,) for Tensor 'input_image:0', which has shape '(?, 224, 224, 3)'

I researched less information about it in Web. I don't know weather codes' locations are wrong or not.
How can i fixed this problems.
Thanks a lot.

[xiaowoniuli]

please tell me the classes is 151 or 1000?.i read the .mat in codes found the classes are 1000 but in the fcn.py the classes are 151.

[zhshining]

I ran and finished FCN.py in CPU Tensorflow based on ubuntu，maybe the reason why my code, copied from https://github.com/shekkizh/FCN.tensorflow/tree/tf.0.11_compatible, can't run successfully is that it is only for ubuntu not windows. I am not sure.

[siddharthamantrala]

Hey @zhshining I am using Windows 8.1 to run the FCN.py and even I am getting the same error you got. Are you sure it works only on Ubuntu? How do I fix this?

Thanks.

[siddharthamantrala]

Hey, @chankeh I got the same error that you got,
Traceback (most recent call last):
File "C:\Users\sidmantrala\Downloads\FCN.tensorflow-master (2)\FCN.tensorflow-master\FCN.py", line 223, in
tf.app.run()
File "C:\Users\sidmantrala\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\platform\app.py", line 126, in run
_sys.exit(main(argv))
File "C:\Users\sidmantrala\Downloads\FCN.tensorflow-master (2)\FCN.tensorflow-master\FCN.py", line 167, in main
train_records, valid_records = scene_parsing.read_dataset(FLAGS.data_dir)
File "C:\Users\sidmantrala\Downloads\FCN.tensorflow-master (2)\FCN.tensorflow-master\read_MITSceneParsingData.py", line 30, in read_dataset
training_records = result['training']
TypeError: 'NoneType' object is not subscriptable

How to fix this? Please help. @shekkizh
Thank You

[chankeh]

@siddharthamantrala you should put the dataset in a right path.

[siddharthamantrala]

@chankeh when i ran the code the first time, it was downloading the code very slowly(unstable internet connection) so i downloaded the net from the browser and put it in the Model_zoo file and i did the same with the Data set and put it in the Data_zoo file.
I put both the file after the directory was created from the code. I just replaced the half downloaded file from the code with the file downloaded from the browser. After doing all that it still shows the same error. Would it affect the Pickle file? Is this method correct......am sorta new to this....please help me.
Thank You

[KKLaLaLa]

hello FKNoble
Sorry to trouble you .
My problem is Cannot feed value of shape (0,) for Tensor 'input_image:0', which has shape '(?, 224, 224, 3)'
what it means
i use tensorflow gpu 1.1.0 windows10

[chankeh]

@siddharthamantrala
put the dataset and model in the path like this,

And i think, if you follow this way,you will run it well.

[siddharthamantrala]

Hey, @chankeh Thanks for the help, the issue got resolved, now am getting another error, I ran the code in Python 3.6.4 CPU Tensorflow in Windows 10 and this is the error am getting:

Setting up Saver...
****************** Epochs completed: 1******************
Traceback (most recent call last):
File "FCN.py", line 223, in
tf.app.run()
File "C:\Users\sidmantrala\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\platform\app.py", line 48, in run
_sys.exit(main(_sys.argv[:1] + flags_passthrough))
File "FCN.py", line 194, in main
sess.run(train_op, feed_dict=feed_dict)
File "C:\Users\sidmantrala\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\client\session.py", line 895, in run
run_metadata_ptr)
File "C:\Users\sidmantrala\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\client\session.py", line 1100, in _run
% (np_val.shape, subfeed_t.name, str(subfeed_t.get_shape())))
ValueError: Cannot feed value of shape (0,) for Tensor 'input_image:0', which has shape '(?, 224, 224, 3)'

This is the same error @zhshining got and @zhshining got it resolved by running it on ubuntu. You seem to be running it on Windows, could you @chankeh please help me on how to resolve this on Windows?

Thank You

[chankeh]

@siddharthamantrala hi , before running the code , you should know that there are so many change in api between tf 0.12 and tf 1.0 . If you want to use 1.0 , you must change some interface in the current code.
yours, sincerely.

[siddharthamantrala]

Hey @chankeh thans for the suggestion, I tried downgrading the tensorflow version to tf.012 and am still getting the same error. Any idea what may be the cause?

Setting up Saver...
****************** Epochs completed: 1******************
Traceback (most recent call last):
File "FCN.py", line 223, in
tf.app.run()
File "C:\Users\sidmantrala\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\platform\app.py", line 48, in run
_sys.exit(main(_sys.argv[:1] + flags_passthrough))
File "FCN.py", line 194, in main
sess.run(train_op, feed_dict=feed_dict)
File "C:\Users\sidmantrala\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\client\session.py", line 895, in run
run_metadata_ptr)
File "C:\Users\sidmantrala\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\client\session.py", line 1100, in _run
% (np_val.shape, subfeed_t.name, str(subfeed_t.get_shape())))
ValueError: Cannot feed value of shape (0,) for Tensor 'input_image:0', which has shape '(?, 224, 224, 3)'

Thank You

[chankeh]

hi @siddharthamantrala ,you should not use the py36,but use the version of py35.
py35 + tf0.12

[zhshining]

@siddharthamantrala sorry, I am busy in preparing my test, so i wouldn't visit my Guihub. I ran it in the Ubuntu successfully, but i don't know why. Now , i plan to try another program.

[lucifer2859]

hi,@chankeh,I use the version of py35 +tf1.6.0 and get the same problem
Traceback (most recent call last):
File "FCN.py", line 222, in
tf.app.run()
File "D:\Anaconda\envs\tensorflow\lib\site-packages\tensorflow\python\platform\app.py", line 126, in run
_sys.exit(main(argv))
File "FCN.py", line 193, in main
sess.run(train_op, feed_dict=feed_dict)
File "D:\Anaconda\envs\tensorflow\lib\site-packages\tensorflow\python\client\session.py", line 905, in run
run_metadata_ptr)
File "D:\Anaconda\envs\tensorflow\lib\site-packages\tensorflow\python\client\session.py", line 1113, in _run
str(subfeed_t.get_shape())))
ValueError: Cannot feed value of shape (0,) for Tensor 'input_image:0', which has shape '(?, 224, 224, 3)'

[qweawq]

hello FKNoble
Sorry to trouble you .
My problem is Cannot feed value of shape (0,) for Tensor 'input_image:0', which has shape '(?, 224, 224, 3)'
what it means
i use tensorflow cpu windows7

[yanzhao-web]

The following error occurred when I was training. Does anyone know how to solve it ?

Caused by op u'inference/deconv2/biases_1', defined at:
File "FCN.py", line 203, in
main()
File "FCN.py", line 158, in main
utils.add_to_regularization_and_summary(var)
File "/home/zhaoyan/Tensorflow/fcn_seg/TensorflowUtils.py", line 227, in add_to_regularization_and_summary
tf.summary.histogram(var.op.name, var)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/summary/summary.py", line 192, in histogram
tag=tag, values=values, name=scope)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/gen_logging_ops.py", line 188, in _histogram_summary
"HistogramSummary", tag=tag, values=values, name=name)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/op_def_library.py", line 787, in _apply_op_helper
op_def=op_def)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/ops.py", line 2956, in create_op
op_def=op_def)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/ops.py", line 1470, in init
self._traceback = self._graph._extract_stack() # pylint: disable=protected-access

InvalidArgumentError (see above for traceback): Nan in summary histogram for: inference/deconv2/biases_1
[[Node: inference/deconv2/biases_1 = HistogramSummary[T=DT_FLOAT, _device="/job:localhost/replica:0/task:0/device:CPU:0"](inference/deconv2/biases_1/tag, inference/deconv2/biases/read/_119)]]
[[Node: batch/fifo_queue_Size/_223 = _Recvclient_terminated=false, recv_device="/job:localhost/replica:0/task:0/device:GPU:0", send_device="/job:localhost/replica:0/task:0/device:CPU:0", send_device_incarnation=1, tensor_name="edge_217_batch/fifo_queue_Size", tensor_type=DT_INT32, _device="/job:localhost/replica:0/task:0/device:GPU:0"]]

[renhaofan]

@chankeh the errors are as follows:
File "FCN.py", line 73, in inference
model_data = utils.get_model_data(FLAGS.model_dir, MODEL_URL)
File "/home/mr/renhaofan/FCN.tensorflow-master/TensorflowUtils.py", line 14, in get_model_data
maybe_download_and_extract(dir_path, model_url)
File "/home/mr/renhaofan/FCN.tensorflow-master/TensorflowUtils.py", line 25, in maybe_download_and_extract
os.makedirs(dir_path)
File "/home/mr/miniconda2/envs/FCN/lib/python2.7/os.py", line 157, in makedirs
mkdir(name, mode)
I think it's because the problem of dataset, can you help me ?

[a123935132]

run FCN.py, this network is only a training version ,I want to know how to directly enter an image for semantic segmentation by this network,can you help me?Thank you very much.

[Asmallmaster]

hi，@lucifer2859.
This code is right. in windows ,the bug is in 'BatchDatsetReader.py'. Note this code in create_image_lists:
filename = os.path.splitext(f.split('/')[-1])[0]
This is no problem in Linux, but in windows, the separator is '\', so you should modify this code to:
filename = os.path.splitext(f.split('\')[-1])[0]
Then delete this file 'MITSceneParsing.pickle', and run read_dataset again.

[Yumi-345]

I meet this problem, too:

ValueError: Cannot feed value of shape (0,) for Tensor 'input_image:0', which has shape '(?, 224, 224, 3)'

and my suggustion is :

look this py
read_MITSceneParsingData.py line 54 ，filename = os.path.splitext(f.split("/")[-1])[0].
we should use ‘\’，not ‘/’ split

because using '/' we can not get the correct filename 'ADE_train_00000001'
for filename is 'ADEChallengeData2016\images\training\ADE_train_00000001'

finnally , i ran it in
win10 ,
python 3.6.3,
tf1.9,
gtx1050ti 4G.
but i change the truct of the net

[iPersevere]

@siddharthamantrala
put the dataset and model in the path like this,

And i think, if you follow this way,you will run it well.

Hello, I've got the same issue as you. Although I do it according to your instruction, it cannot be solved. Any idea what may be cause?

[1996lcc]

Hi！
I have this problem ”No module named 'TensorflowUtils'“at runtimeI run“FCN.py".
I've tried"pip install tensorflow-utils'.

[chenchunyulumi]

Hello,I've got a new problem
Traceback (most recent call last):
File "FCN.py", line 230, in
tf.app.run()
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/platform/app.py", line 48, in run
_sys.exit(main(_sys.argv[:1] + flags_passthrough))
File "FCN.py", line 167, in main
train_records, valid_records = scene_parsing.read_dataset(FLAGS.data_dir)
File "/home/hk/下载/FCN.tensorflow-master/read_MITSceneParsingData.py", line 30, in read_dataset
training_records = result['training']
TypeError: 'NoneType' object has no attribute 'getitem'
Could anyone solve this problem?
thank you very much

[ghost]

Hi !
I have a problem ,the code has worked in my computer,but the loss value ramain 3 , it is Never convergence. Any idea what may be cause?

[zohaibjan]

I meet this problem, too:

ValueError: Cannot feed value of shape (0,) for Tensor 'input_image:0', which has shape '(?, 224, 224, 3)'

and my suggustion is :

look this py
read_MITSceneParsingData.py line 54 ，filename = os.path.splitext(f.split("/")[-1])[0].
we should use ‘\’，not ‘/’ split

because using '/' we can not get the correct filename 'ADE_train_00000001'
for filename is 'ADEChallengeData2016\images\training\ADE_train_00000001'

finnally , i ran it in
win10 ,
python 3.6.3,
tf1.9,
gtx1050ti 4G.
but i change the truct of the net

That is because the data cannot be loaded. Make sure you change / to \\ if you are running it on windows

[ShikunChen]

我也遇到这个问题：
ValueError：无法为张量为'（？，224，224，3）'的张量'input_image：0'输入形状（0，）的值
我的建议是：
看看这个py
read_MITSceneParsingData.py第54行，文件名= os.path.splitext（f.split（“ /”）[-1]）[0]。
我们应该使用''，而不是'/'split
因为使用'/'我们无法获得正确的文件名'ADE_train_00000001'，
因为文件名是'ADEChallengeData2016 \ images \ training \ ADE_train_00000001'
最后，我在
win10，
python 3.6.3，
tf1.9，
gtx1050ti 4G中运行了它。
但是我改变了网的结构

这是因为无法加载数据。如果在Windows上运行，请确保将/更改为\
In addition, please make sure the image name corresponds

[201922010202]

我用的是0.12rc版本，但是出现了这样的情况：

回溯（最近一次调用最后一次）：
找到泡菜文件！
文件“E:/Python/FCN/FCN.py”，第 224 行，在
tf.app.run()
文件“D:\Program Files\Anaconda3\lib\site-packages\tensorflow\python\platform\app.py "，第 43 行，运行中
sys.exit(main(sys.argv[:1] + flags_passthrough))
文件“E:/Python/FCN/FCN.py”，第 168 行，在主
train_records 中，valid_records = scene_parsing.read_dataset (FLAGS.data_dir)
File "E:\Python\FCN\read_MITSceneParsingData.py", line 30, in read_dataset
training_records = result['training']
TypeError: 'NoneType' object is not subscriptable

我怎样才能解决这个问题？
谢谢。