以下是我multiple records( around 5000-7000)使用Prepared Statement插入到Oracle数据库中的代码。
multiple records( around 5000-7000)
我目前的工作方式好吗?还是可以使用某些方法来进一步改进batch thing?
batch thing
pstatement = db_connection.prepareStatement(PDSLnPConstants.UPSERT_SQL); for (Entry<Integer, LinkedHashMap<Integer, String>> entry : MAPPING.entrySet()) { pstatement.setInt(1, entry.getKey()); pstatement.setString(2, entry.getValue().get(LnPConstants.CGUID_ID)); pstatement.setString(3, entry.getValue().get(LnPConstants.PGUID_ID)); pstatement.setString(4, entry.getValue().get(LnPConstants.SGUID_ID)); pstatement.setString(5, entry.getValue().get(LnPConstants.UID_ID)); pstatement.setString(6, entry.getValue().get(LnPConstants.ULOC_ID)); pstatement.setString(7, entry.getValue().get(LnPConstants.SLOC_ID)); pstatement.setString(8, entry.getValue().get(LnPConstants.PLOC_ID)); pstatement.setString(9, entry.getValue().get(LnPConstants.ALOC_ID)); pstatement.setString(10, entry.getValue().get(LnPConstants.SITE_ID)); pstatement.executeUpdate(); pstatement.clearParameters(); }
我正在使用的更新的代码:-
public void runNextCommand() { Connection db_connection = null; PreparedStatement pstatement = null; int batchLimit = 1000; boolean autoCommit = false; try { db_connection = getDBConnection(); autoCommit = db_connection.getAutoCommit(); db_connection.setAutoCommit(false); //Turn off autoCommit pstatement = db_connection.prepareStatement(LnPConstants.UPSERT_SQL); // create a statement for (Entry<Integer, LinkedHashMap<Integer, String>> entry : GUID_ID_MAPPING.entrySet()) { pstatement.setInt(1, entry.getKey()); pstatement.setString(2, entry.getValue().get(LnPConstants.CGUID_ID)); pstatement.setString(3, entry.getValue().get(LnPConstants.PGUID_ID)); pstatement.setString(4, entry.getValue().get(LnPConstants.SGUID_ID)); pstatement.setString(5, entry.getValue().get(LnPConstants.UID_ID)); pstatement.setString(6, entry.getValue().get(LnPConstants.ULOC_ID)); pstatement.setString(7, entry.getValue().get(LnPConstants.SLOC_ID)); pstatement.setString(8, entry.getValue().get(LnPConstants.PLOC_ID)); pstatement.setString(9, entry.getValue().get(LnPConstants.ALOC_ID)); pstatement.setString(10, entry.getValue().get(LnPConstants.SITE_ID)); pstatement.addBatch(); batchLimit--; if(batchLimit == 0){ pstatement.executeBatch(); pstatement.clearBatch(); batchLimit = 1000; } pstatement.clearParameters(); } } catch (SQLException e) { getLogger().log(LogLevel.ERROR, e); } finally { try { pstatement.executeBatch(); db_connection.commit(); db_connection.setAutoCommit(autoCommit); } catch (SQLException e1) { getLogger().log(LogLevel.ERROR, e1.getMessage(), e1.fillInStackTrace()); } if (pstatement != null) { try { pstatement.close(); pstatement = null; } catch (SQLException e) { getLogger().log(LogLevel.ERROR, e.getMessage(), e.fillInStackTrace()); } } if (db_connection!= null) { try { db_connection.close(); db_connection = null; } catch (SQLException e) { getLogger().log(LogLevel.ERROR, e.getMessage(), e.fillInStackTrace()); } } } }
您可以考虑addBatch()一次性使用和执行语句背面。另外,正如@pst在您的问题中指出的那样,请考虑使用trasaction。
addBatch()
trasaction
您将执行以下操作:
boolean autoCommit = connection.getAutoCommit(); try{ connection.setAutoCommit(false //Turn off autoCommit pstatement = db_connection.prepareStatement(PDSLnPConstants.UPSERT_SQL); int batchLimit = 1000; try{ for (Entry<Integer, LinkedHashMap<Integer, String>> entry : MAPPING.entrySet()) { pstatement.setInt(1, entry.getKey()); pstatement.setString(2, entry.getValue().get(LnPConstants.CGUID_ID)); pstatement.setString(3, entry.getValue().get(LnPConstants.PGUID_ID)); pstatement.setString(4, entry.getValue().get(LnPConstants.SGUID_ID)); pstatement.setString(5, entry.getValue().get(LnPConstants.UID_ID)); pstatement.setString(6, entry.getValue().get(LnPConstants.ULOC_ID)); pstatement.setString(7, entry.getValue().get(LnPConstants.SLOC_ID)); pstatement.setString(8, entry.getValue().get(LnPConstants.PLOC_ID)); pstatement.setString(9, entry.getValue().get(LnPConstants.ALOC_ID)); pstatement.setString(10, entry.getValue().get(LnPConstants.SITE_ID)); pstatement.addBatch(); batchLimit--; if(batchLimit == 0){ pstatement.executeBatch(); pstatement.clearBatch batchLimit = 1000; } pstatement.clearParameters(); } }finally{ //for the remaining ones pstatement.executeBatch(); //commit your updates connection.commit(); } }finally{ connection.setAutoCommit(autoCommit); }
这个想法是为批处理更新设置一个限制,并且仅在达到特定限制时才执行数据库更新。这样,您就可以将数据库调用限制为每次batchLimit定义一次。这样,它将更快。
batchLimit
另请注意transaction,我已经展示了如何以及何时使用commit。这可能并不总是正确的指向,commit因为此决定将基于您的要求。您可能还想rollback在发生异常的情况下执行。因此,由您决定。
transaction
commit
rollback
查看“使用交易”教程,以更好地了解如何使用transaction。