Is there a way to trigger a method in a Spider class just before it terminates?
I can terminate the spider myself, like this:
class MySpider(CrawlSpider): #Config stuff goes here... def quit(self): #Do some stuff... raise CloseSpider('MySpider is quitting now.') def my_parser(self, response): if termination_condition: self.quit() #Parsing stuff goes here...
But I can't find any information on how to determine when the spider is about to quit naturally.