我正在使用cdh5 quickstart vm,并且我有一个像这样的文件(此处不完整):
{"user_id": "kim95", "type": "Book", "title": "Modern Database Systems: The Object Model, Interoperability, and Beyond.", "year": "1995", "publisher": "ACM Press and Addison-Wesley", "authors": {}, "source": "DBLP" } {"user_id": "marshallo79", "type": "Book", "title": "Inequalities: Theory of Majorization and Its Application.", "year": "1979", "publisher": "Academic Press", "authors": {("Albert W. Marshall"), ("Ingram Olkin")}, "source": "DBLP" }
我使用了这个脚本:
books = load 'data/book-seded.json' using JsonLoader('t1:tuple(user_id: chararray,type:chararray,title:chararray,year:chararray,publisher:chararray,source:chararray,authors:bag{T:tuple(author:chararray)})'); STORE books INTO 'book-no-seded.tsv';
该脚本有效,但是生成的文件为空,您有什么想法吗?
最后,只有此模式有效:如果我添加或删除与该配置不同的空格,那么我将出现错误(我还为元组添加了“名称”,并在其为空时指定了“ null”,并更改了作者之间的顺序和来源,但即使没有这种配置,它仍然是错误的)
{"user_id": "kim95", "type": "Book","title": "Modern Database Systems: The Object Model, Interoperability, and Beyond.", "year": "1995", "publisher": "ACM Press and Addison-Wesley", "authors": [{"name":null"}], "source": "DBLP"} {"user_id": "marshallo79", "type": "Book", "title": "Inequalities: Theory of Majorization and Its Application.", "year": "1979", "publisher": "Academic Press", "authors": [{"name":"Albert W. Marshall"},{"name":"Ingram Olkin"}], "source": "DBLP"}
工作脚本是这个:
books = load 'data/book-seded-workings-reduced.json' using JsonLoader('user_id:chararray,type:chararray,title:chararray,year:chararray,publisher:chararray,authors:{(name:chararray)},source:chararray'); STORE books INTO 'book-table.csv'; //whether .tsv or .csv