小编典典

如何在Elasticsearch提示器中使用Logstash导入CSV数据以完成字段类型

elasticsearch

ElasticSearch索引创建

curl -XPOST 'http://localhost:9200/music/' -d '{}'

场图

curl -XPUT 'http://localhost:9200/music/_mapping/song' -d '
{
  "properties": {
    "name" : {
      "type" : "string"
    },
    "suggest": {
      "type" : "completion"
    }
  }
}'

LogStash配置文件musicStash.config

input {
    file {
        path => "pathToCsv"
        start_position => beginning
    }
}

filter {  
    csv {
        columns => ["id", "name", "suggest"]
        separator => ","
    }
}

output {
    elasticsearch {
        hosts => "localhost"
        index => "music"
        document_id => "%{id}"
    }
}

现在,在执行logstash配置文件时,在elasticsearch控制台中收到以下异常

failed to put mappings on indices [[music]], type [logs]
java.lang.IllegalArgumentException: Mapper for [suggest] conflicts with existing mapping in other types:
[mapper [suggest] cannot be changed from type [completion] to [string]]
at org.elasticsearch.index.mapper.FieldTypeLookup.checkCompatibility(FieldTypeLookup.java:117)

在Logstash控制台中收到错误,

response=>{"index"=>{"_index"=>"music", "_type"=>"logs", "_id"=>"5", "status"=>400, 
"error"=>{"type"=>"illegal_argument_exception", 
"reason"=>"Mapper for [suggest] conflicts with existing mapping in other types:\n[mapper [suggest] cannot be changed from type [completion] to [string]]"}}}, :level=>:warn}

那么如何通过Logstash导入csv文件来实现elasticsearch自动完成功能。


阅读 762

收藏
2020-06-22

共1个答案

小编典典

您在elasticsearch输出中缺少以下设置:

document_type => "song"

发生的是,logstash创建了一个新类型logs默认情况下),并且由于从ES 2.0版本开始,它被禁止在同一个索引中包含两个具有相同名称但不同类型(stringvs
completion)的字段,因此会出错。

只需像这样修改您的输出,它将起作用:

output {
    elasticsearch {
        hosts => "localhost"
        index => "music"
        document_type => "song"
        document_id => "%{id}"
    }
}
2020-06-22