Allow spiders to also return/yield ItemLoader.

[vincent-ferotin]

Behind, Scrapy will convert ItemLoader to Item by calling
:meth:ItemLoader.load_item.

[whalebot-helmsman]

@vincent-ferotin Problem with unit-tests was fixed in master. Can you merge master to this PR?

[codecov]

Codecov Report

Merging #3244 into master will decrease coverage by 0.01%.
The diff coverage is 25%.

@@            Coverage Diff            @@
##           master   #3244      +/-   ##
=========================================
- Coverage   82.12%   82.1%   -0.02%     
=========================================
  Files         228     228              
  Lines        9599    9602       +3     
  Branches     1385    1386       +1     
=========================================
+ Hits         7883    7884       +1     
- Misses       1457    1458       +1     
- Partials      259     260       +1

Impacted Files	Coverage Δ
scrapy/core/scraper.py	85.33% <25%> (-1.07%)	⬇️

[vincent-ferotin]

@whalebot-helmsman Here it is, I hope so? (not enough familiar/confident with git/GitHub to be sure, please let tell me if it is not ok for you ^^;)

[whalebot-helmsman]

Thanks, @vincent-ferotin mentioned problems with twisted is gone. I don't know the reason why other tests are in failed state.

[kmike]

Hi @vincent-ferotin! I can see how this feature can save some typing, and API improvements are good to have. However, currently I'm -0 to add such feature because:

• there was an idea to move ItemLoaders out of Scrapy;
• it complicates API of a spider; custom middlewares would need to be modified in order to support a new possible output type (?).

The implementation also needs work:

• if I'm not mistaken, currently it can lead to subtle bugs (see ItemLoader should work with copy of item passed as an argument #616)
• there are no tests;
• documentation is not updated (you can probably check Allow spiders to return dicts.  #1081 for suggestions of doc locations to be updated).

[Gallaecio]

Closing as, even if implemented, https://github.com/scrapy/itemadapter would be the right place now.