所以说我们有两个模型
class Product(models.Model): """ A model representing a product in a website. Has new datapoints referencing this as a foreign key daily """ name = models.CharField(null=False, max_length=1024, default="To be Scraped") url = models.URLField(null=False, blank=False, max_length=10000) class DataPoint(models.Model): """ A model representing a datapoint in a Product's timeline. A new one is created for every product daily """ product = models.ForeignKey(Product, null=False) price = models.FloatField(null=False, default=0.0) inventory_left = models.BigIntegerField(null=False, default=0) inventory_sold = models.BigIntegerField(null=False, default=0) date_created = models.DateField(auto_now_add=True) def __unicode__(self): return "%s - %s" % (self.product.name, self.inventory_sold)
目标是根据产品附加的最新数据点的ventory_sold值对产品的QuerySet进行排序。这是我到目前为止的内容:
products = Product.objects.all() datapoints = DataPoint.objects.filter(product__in=products) datapoints = list(datapoints.values("product__id", "inventory_sold", "date_created")) products_d = {} # Loop over the datapoints values array for i in datapoints: # If a datapoint for the product doesn't exist in the products_d, add the datapoint if str(i["product__id"]) not in products_d.keys(): products_d[str(i["product__id"])] = {"inventory_sold": i["inventory_sold"], "date_created": i["date_created"]} # Otherwise, if the current datapoint was created after the existing datapoint, overwrite the datapoint in products_d else: if products_d[str(i["product__id"])]["date_created"] < i["date_created"]: products_d[str(i["product__id"])] = {"inventory_sold": i["inventory_sold"], "date_created": i["date_created"]} # Sort the products queryset based on the value of inventory_sold in the products_d dictionary products = sorted(products, key=lambda x: products_d.get(str(x.id), {}).get("inventory_sold", 0), reverse=True)
这可以正常工作,但是对于大量(500,000〜)的产品和数据点来说,它的运行速度非常慢。有没有更好的方法可以做到这一点?
顺便提一句(不重要),由于我无法找到任何有关此的信息,因此DataPoint模型的unicode方法似乎也在进行不必要的SQL查询。Django模型传递给模板后,这是否是Django模型的默认特征?
我认为您可以在此处使用子查询来注释最新数据点的值,然后对其进行排序。
根据这些文档中的示例,结果将类似于:
from django.db.models import OuterRef, Subquery newest = DataPoint.objects.filter(product=OuterRef('pk')).order_by('-date_created') products = Product.objects.annotate( newest_inventory_sold=Subquery(newest.values('inventory_sold')[:1]) ).order_by('newest_inventory_sold')
从侧面讲,为避免在输出DataPoints时出现额外的查询,您将需要select_related在原始查询中使用:
select_related
datapoints = DatePoint.objects.filter(...).select_related('product')
这将执行JOIN操作,以便获取产品名称不会引起新的数据库查找。