Later on Server B ( better performance ) , found in a batch to save the database where there OutOfMemory, the feeling is not the number of threads and more, a little less , the test in the end, found only with a thread ( which is the default one , at most two ) , so as to run in the past , which is good because the machine performance , multi-threaded execution saved in fast causing the problem ? What are the machine's memory 100G +, cpu is very cattle, how will the memory overflow ? Or start jvm to have enough memory ?
great God who is an analysis please , thank you .
------ Solution ---------------------------------------- ----
count your 200 a data fields, each 8 bytes , 10000 bytes plus a text , only the 5000 ( 200 * 8 + 10000 ) * 5000 = 58MB, 3 threads Cheng Si less than 200MB of memory. Check your JVM startup memory it
------ Solution --------------------------------- -----------
I forgot to say one thing : change the program in two parts, a ) to read the file data, and be saved to the database ; 2 ) reads the database for processing ( here only relates to the operation of the database ) . In fact , multi-threading occurs only in 2 ) inside, it has nothing to do with the file read . In addition, 2 ) which reads 5000 per data into a thread for processing , there have batch save data ( < = 5000 ) action , OutOfMemory is what has happened here .
should be you query the database , and the resulting amount of data exceeds the memory size of your boot . tomcat can modify catalina.sh startup memory size ( default is like 80M), other container configuration is not clear.
------ Solution ---------------------------------------- ----
not set jvm memory it
------ Solution --------------------------- -----------------
machine performance is good cause memory overflow ? kidding , what relationship does not.
bad performance of your machine will not necessarily overflow.
jvm memory overflow check allocated memory is not small. additional tracking what unit of time the amount of data into the database .
------ Solution ----------- ---------------------------------
checking code vulnerabilities ; transfer large the JVM -Xmx and Xms, let JVM has more available memory space .
------ Solution ---------------------------------------- ----
read data size exceeds the jvm memory limit , what is the relationship with the machine .
------ Solution ---------------------------------------- ----
memory there are two possibilities , jvm memory ( At this point you can try upstairs method ) , the unit of memory ( file stream operations to consider whether it is a problem )
--- --- Solution --------------------------------------------
OutOfMemoryError causes is because you loaded in memory a lot of data , it is estimated you read all the files of a List or something similar to the data structure , and if the file is too large number of rows , the final only such a result !
You have to read some data operations to complete before the second batch of data to read , so you can control the memory usage within a reasonable range, and almost impossible to appear OutOfMemoryError errors !
so much for such a program in terms of memory is overkill , it should be optimized from the application itself , rather than simply rely on increasing the server's hardware configuration.
------ Solution ---------------------------------------- ----
the question is : landlord program , no control memory usage.
solutions
will generally set the maximum JVM memory usage values, but also the risk of a memory overflow .
is best able to optimize program code, for example , re-use buffers , arrays , etc., to minimize the creation of various objects .
Also, the collection object , after use , to call clear methods.
------ For reference only -------------------------------------- -
theoretically possible . You to read the document too fast over the database operations .
Of course , the code design should also be flawed . Can be considered, if the queue in the number of files to be processed much more than one limit , it no longer reads the document.
------ For reference only -------------------------------------- -
may still program their own problems, optimize what the program should be able to solve this problem.
------ For reference only -------------------------------------- -
machine , then the landlord you look eclipse configuration eclipse should be a configuration problem .
For the program , then to be optimized
remember watching "effective java" when inside said , obviously some programs you run on your computer in your own good , until the publishing process to the user when a variety of errors have emerged .
------ For reference only -------------------------------------- -
memory management are Java programmers weaknesses.
First of all, your machine has more than 100 G of memory, the JVM does not mean you can use , here for the JVM debugging effort to the next point .
Secondly, a floor that you read the file too soon , too late to write to the database and then release the memory , this is your program design flaws. Never heard of good programmers complain about machine performance can not cause the program to run ( historically also have such cases, such as some of the early games , most notably Red Alert 95 , machine performance is good, your game will in an end within seconds , it has no internal clock as the CPU clock speed thanks to standard , CPU a fast , it embarrassing ) . This time you have to do is limit the file size of the read buffer , full of not reading, processing finished before continuing to read
------ For reference only ---------------------------------- -----
I forgot to say one thing : change the program in two parts, a ) to read the file data, and be saved to the database ; 2 ) reads the database for processing ( here only relates to the operation of the database ) . In fact , multi-threading occurs only in 2 ) inside, it has nothing to do with the file read . In addition, 2 ) which reads 5000 per data into a thread for processing , there have batch save data ( < = 5000 ) action , OutOfMemory is what has happened here .
------ For reference only -------------------------------------- -
------ For reference only ---------------------------------- -----
may be positive solution
------ For reference only ------------------------------- --------
may be positive solution
big cow upstairs c + + to java edition also mixed fraction to leave your old spare scores to give us these dishes it
------ For reference only ---- -----------------------------------
tune JVM memory is not a solution , or optimize the code it,
------ For reference only -------------------------------------- -
ah pack , you come back T T
------ For reference only --------------------- ------------------
may be positive solution
big cow upstairs c + + to java edition also mixed fraction to leave your old spare us these dishes it scores ranked 92,
------ For reference only ---------------------------- -----------
I go, there is a big cow ,
------ For reference only -------------- -------------------------
set . . ------ For reference only -------------------------------------- -
actually a very simple issue , deploy the guy did not change the JVM -Xmx and Xms, was he mad ! Necessarily to a lot of large cattle , happy ah ~ ~
scattered points
没有评论:
发表评论