Here's a little trouble.

[peterxiashidage]

Nice code! However, In 2023.11, When I'm trying to write a spider with selenium, I find that the data is not saved in one page. So I have to harvest data page by page!

[shujuecn]

Nice code! However, In 2023.11, When I'm trying to write a spider with selenium, I find that the data is not saved in one page. So I have to harvest data page by page!

Thank you for the feedback. TCMSP may have had some updates since the original code was written that I did not anticipate. Without seeing your specific code it is difficult to pinpoint what the issue may be. I would suggest open sourcing your version so the community can collectively learn from it and improve upon it. Let me know if you need any other suggestions on how to proceed, and I'm happy to help however I can.

[peterxiashidage]

Nice code! However, In 2023.11, When I'm trying to write a spider with selenium, I find that the data is not saved in one page. So I have to harvest data page by page!

Thank you for the feedback. TCMSP may have had some updates since the original code was written that I did not anticipate. Without seeing your specific code it is difficult to pinpoint what the issue may be. I would suggest open sourcing your version so the community can collectively learn from it and improve upon it. Let me know if you need any other suggestions on how to proceed, and I'm happy to help however I can.

Sorry to the delayed reply! I think I do managed to harvest data through Beatifulsoup4, tough I still can't figure out why locating target method with Xpath failed. Thanks for your help!