如果嵌套字典前面没有键,我现在可以删除重复项。我可以使用此功能的字典列表的一个示例是:
[{'asndb_prefix': '164.39.xxx.0/17', 'cidr': '164.39.xxx.0/17', 'cymru_asn': 'XXX', 'cymru_country': 'GB', 'cymru_owner': 'XXX , GB', 'cymru_prefix': '164.39.xxx.0/17', 'ips': ['164.39.xxx.xxx'], 'network_id': '164.39.xxx.xxx/24',}, {'asndb_prefix': '54.192.xxx.xxx/16', 'cidr': '54.192.0.0/16', 'cymru_asn': '16509', 'cymru_country': 'US', 'cymru_owner': 'AMAZON-02 - Amazon.com, Inc., US', 'cymru_prefix': '54.192.144.0/22', 'ips': ['54.192.xxx.xxx', '54.192.xxx.xxx'], 'network_id': '54.192.xxx.xxx/24', }] def remove_dict_duplicates(list_of_dicts): """ "" Remove duplicates in dict """ list_of_dicts = [dict(t) for t in set([tuple(d.items()) for d in list_of_dicts])] # remove the {} before and after - not sure why these are placed as # the first and last element return list_of_dicts[1:-1]
但是,我希望能够根据键和该词典中关联的所有值删除重复项。因此,如果内部有相同的键但值不同,则我不想删除它,但是如果有完整的副本,则将其删除。
[{'50.16.xxx.0/24': {'asndb_prefix': '50.16.0.0/16', 'cidr': '50.16.0.0/14', 'cymru_asn': 'xxxx', 'cymru_country': 'US', 'cymru_owner': 'AMAZON-AES - Amazon.com, Inc., US', 'cymru_prefix': '50.16.0.0/16', 'ip': '50.16.221.xxx', 'network_id': '50.16.xxx.0/24', 'pyasn_asn': xxxx, 'whois_asn': 'xxxx'}}, // This would be removed {'50.16.xxx.0/24': {'asndb_prefix': '50.16.0.0/16', 'cidr': '50.16.0.0/14', 'cymru_asn': 'xxxxx', 'cymru_country': 'US', 'cymru_owner': 'AMAZON-AES - Amazon.com, Inc., US', 'cymru_prefix': '50.16.0.0/16', 'ip': '50.16.221.xxx', 'network_id': '50.16.xxx.0/24', 'pyasn_asn': xxxx, 'whois_asn': 'xxxx'}}, // This would NOT be removed {'50.16.xxx.0/24': {'asndb_prefix': '50.999.0.0/16', 'cidr': '50.999.0.0/14', 'cymru_asn': 'xxxx', 'cymru_country': 'US', 'cymru_owner': 'AMAZON-AES - Amazon.com, Inc., US', 'cymru_prefix': '50.16.0.0/16', 'ip': '50.16.221.xxx', 'network_id': '50.16.xxx.0/24', 'pyasn_asn': xxxx, 'whois_asn': 'xxxx'}}]
我该怎么做呢?谢谢。
要从字典列表中删除重复项,请执行以下操作:
list_of_unique_dicts = [] for dict_ in list_of_dicts: if dict_ not in list_of_unique_dicts: list_of_unique_dicts.append(dict_)