我有内存问题,因为看起来Django在使用delete()时将对象加载到内存中.有没有办法阻止Django这样做？
来自Django文档：
Django needs to fetch objects into memory to send signals and handle cascades. However,if there are no cascades and no signals,then Django may take a fast-path and delete objects without fetching into memory. For large deletes this can result in significantly reduced memory usage. The amount of executed queries can be reduced,too.
https://docs.djangoproject.com/en/1.8/ref/models/querysets/#delete
我不使用信号.我在我试图删除的模型上有外键,但我不明白为什么Django需要将对象加载到内存中.它看起来确实如此,因为我的内存随着查询的运行而上升.
解决方法
您可以使用这样的函数迭代大量对象而不使用太多内存：
import gc
def queryset_iterator(qs,batchsize = 500,gc_collect = True):
iterator = qs.values_list('pk',flat=True).order_by('pk').distinct().iterator()
eof = False
while not eof:
primary_key_buffer = []
try:
while len(primary_key_buffer) < batchsize:
primary_key_buffer.append(iterator.next())
except StopIteration:
eof = True
for obj in qs.filter(pk__in=primary_key_buffer).order_by('pk').iterator():
yield obj
if gc_collect:
gc.collect()
for obj in queryset_iterator(HugeQueryset.objects.all()):
obj.delete()
有关更多信息,请查看this blog post.
