我正在尝试使用Logstash将XML转换为ElasticSearch的JSON。我能够读取值并将其发送到ElasticSearch。问题是所有值都以数组形式出现。我想让它们像字符串一样出来。我知道我可以replace为每个字段分别执行操作,但是随后遇到了一个问题,嵌套字段的深度为3级。
replace
XML格式
<?xml version="1.0" encoding="UTF-8" standalone="yes"?> <acs2:SubmitTestResult xmlns:acs2="http://tempuri.org/" xmlns:acs="http://schemas.sompleace.org" xmlns:acs1="http://schemas.someplace.org"> <acs2:locationId>Location Id</acs2:locationId> <acs2:userId>User Id</acs2:userId> <acs2:TestResult> <acs1:CreatedBy>My Name</acs1:CreatedBy> <acs1:CreatedDate>2015-08-07</acs1:CreatedDate> <acs1:Output>10.5</acs1:Output> </acs2:TestResult> </acs2:SubmitTestResult>
Logstash配置
input { file { path => "/var/log/logstash/test.xml" } } filter { multiline { pattern => "^\s\s(\s\s|\<\/acs2:SubmitTestResult\>)" what => "previous" } if "multiline" in [tags] { mutate { replace => ["message", '<?xml version="1.0" encoding="UTF-8" standalone="yes"?>%{message}'] } xml { target => "SubmitTestResult" source => "message" } mutate { remove_field => ["message", "@version", "host", "@timestamp", "path", "tags", "type"] remove_field => ["entry", "[SubmitTestResult][xmlns:acs2]", "[SubmitTestResult][xmlns:acs]", "[SubmitTestResult][xmlns:acs1]"] # This works replace => [ "[SubmitTestResult][locationId]", "%{[SubmitTestResult][locationId]}" ] # This does NOT work replace => [ "[SubmitTestResult][TestResult][CreatedBy]", "%{[SubmitTestResult][TestResult][CreatedBy]}" ] } } } output { stdout { codec => "rubydebug" } elasticsearch { index => "xmltest" cluster => "logstash" } }
示例输出
{ "_index": "xmltest", "_type": "logs", "_id": "AU8IZBURkkRvuur_3YDA", "_version": 1, "found": true, "_source": { "SubmitTestResult": { "locationId": "Location Id", "userId": [ "User Id" ], "TestResult": [ { "CreatedBy": [ "My Name" ], "CreatedDate": [ "2015-08-07" ], "Output": [ "10.5" ] } ] } } }
如您所见,输出是每个元素的数组(我替换为的locationId除外)。我试图不必对每个元素进行替换。有没有办法调整配置以使输出正确放置?如果没有,我该如何深入3个等级replace?
-更新-
我想出了如何在测试结果中达到第三级的方法。替换为:
replace => [ "[SubmitTestResult][TestResult][0][CreatedBy]", "%{[SubmitTestResult][TestResult][0][CreatedBy]}" ]
我想到了。这是解决方案。