then asked a group of experienced people in , that may be loaded in memory is too large. Then I saw them, I went to this little broken table actually accounted 6G. . Then I added some filters , about 4G look.
then press the Internet, said the tomcat Xmx get 4500m. . . . Then hopefully start to see the log , I went beyond limits . Baffled ah , but also to ask people . Let people come back to see if the machine is a bit much , said 32 of the JAVA virtual machine to the maximum amount of memory seems to only open about 2G. 64G can be large . Then I saw , it really is 32-bit . . . .
Then I have no other way . . . Ladies and gentlemen, who can explain to me why tomcat according to 32 , 64 to limit the size of the JAVA virtual machine . Machines have 12G memory , idle there 10G. . . But not even support . Not really depressed , and then use the interface to access a servlet , they reported NullPointerException, I guess it is because the data is not loaded into memory , this interface an investigation found that the data is empty, then threw an exception. ( This is my guess, I hope you can give me a judgment based on experience . )
Thank you . . Who can give better suggestions , tomorrow to discuss the matter with the mentor , another 64 machines to carry out his ~ ~
------ Solution ------------------ --------------------------
first thing you put table are loaded into memory , which in itself unreasonable. Can not be paged query ?
Second 32-bit operating system that supports memory to 4G, which has occupied part of the memory system , so your program can be used less memory . And if a 32-bit system , you insert 8G/16G memory, the memory can identify no more than 4G.
------ Solution ---------------------------------------- ----
this problem from your perspective, even gave you 64 system may not be able to solve your problem.
such a large amount of data loaded into memory
All the HashMap , this design is definitely a problem.
HashMap Although optimized hash value through rapid positioning , but in the final analysis HashMap way or traverse positioning , positioning a small amount of data quickly, but the large amount of data will be very poor performance , the performance is far from the index database Btree like structure used to the quick search . So you may want to optimize the system , to optimize the results of the system to the pit .
------ Solution ---------------------------------------- ----
I wonder why the company wants you to fully understand this man to optimize system design projects . See you on the architecture of memory and understanding , can not seem to finish the job ...
a 6GB data tables into memory is not so - . And many times, do not need this kind of optimization, indexing, and usually just some optimization logic can apply on basically all business systems. If concurrency is really large, huge traffic , it is too frequently the data apply it to the support and well-designed hardware , even if the data are frequently used , it is impossible to 6GB of data very frequently - as long as the statistics which 6GB cache frequently then you can, like garbage collection mechanism of java .
If the data analysis system , I can tell you , based on your current hardware configuration can not complete the task, of course, also need a specific design and a lot of code optimization , you're not in the same level able to complete , requires a very experienced on a large data processing experienced people to deal with.
------ For reference only -------------------------------------- -
you too pulled , the operating system you are 32 , how to support 12G memory ah.
32 -bit operating system will only support the largest 4G memory ah. This is based on the operating system memory addressing relevant . For example, a pointer to a memory address represents a 32 with space to store , so certainly can only support so much memory ah. Learned of the operating system should be understood ah.
------ For reference only -------------------------------------- -
32 4G is a power of 2 , it is recommended to find a business consulting agency TAP then deal with it
------ For reference only ------------- --------------------------
tonight met a , PS installed when prompted to install 64 would be more Some handy .
machine decided it is estimated that the maximum memory inside the JVM settings 32 and 64 occupied different.
------ For reference only -------------------------------------- -
top ~
also saw two days before the relevant content , probably meaning that 32-bit operating system , it is the two largest addressable memory space of 32 th , the memory allocation more on the out of range, the excess or laid hold of . Therefore , jdk or tomcat initialize memory size in check when the school system Examination median ( this is the inference ) .
------ For reference only -------------------------------------- -
why you want so much data is loaded into memory?
------ For reference only -------------------------------------- -
third-party caching plugin can be used. If memcache
------ For reference only ------------------------------------ ---
a table stand 6 G I think the landlord gave up the idea of memory to put it right !
------ For reference only -------------------------------------- -
64 -bit it looks like the biggest landlord 8G memory optimization program should have changed under
------ For reference only -------- -------------------------------
I was also wondering not ah. . . Look machine obviously 12G memory. But indeed the 32 's .
I corrected it, 32 not only support the 4G . If you turn on PAE , the biggest support 64G. Only one process can only address 4G of memory. So , tomcat as a process , you can not use more than 4G space. May be the reason for it. .
------ For reference only -------------------------------------- -
this makes sense. Thank
------ For reference only ------------------------------------- -
64 words , 64 th power . Not a simple superposition .
------ For reference only -------------------------------------- -
64 words , 64 th power . Not a simple superposition .
I committed two of the . . . 64 The theoretical value is great , but the hardware restrictions ah. . .
------ For reference only -------------------------------------- -
java virtual machine with system memory size , generally 64 is greater than 32 , linux system is greater than the window system
------ For reference only ---------- -----------------------------
still can not comprehend such a large data into memory. If you want to optimize , you can first consider the current database query optimization , or handle it from scratch to deal with logic.
I'm also a 32-bit machine , 4G memory , can not have used realistic hi3.17G available.
------ For reference only -------------------------------------- -
so much data is read into memory , nor the reality of it
------ For reference only --------------------- ------------------
32 -bit system , opened the PAE does have more than 4G of memory usage can be displayed , but you tomcat using JDK should be 32 , In addition, it is unclear which opened a PAE of 32 system supports 64-bit JDK, estimates are not supported .
64 -bit system , you install 32-bit JDK, I have the impression that the most they can with 4G, but if it is windows system , it is likely to have less than 4G , it should be directly on the mountains tomcat
landlord your needs , you may need a search engine to improve performance
------ For reference only ------------------- --------------------
ah , like to say next . Solve the problem, is not taken into account another place caused a null pointer .
give you a suggestion is to look carefully to see posts point . .
I clearly say is:
not really depressed , and then use servlet access to an interface , they reported NullPointerException, I guess it is because the data is not loaded into memory , this interface an investigation found that the data is empty, then threw an exception. ( This is my guess, I hope you can give me a judgment based on experience ..)
So, if experienced should know right away is not a memory problem, but what throws an exception . So, with a big cow on the Internet describe it, he immediately said, nothing to do with memory , null pointer is your problem . Then I found out later it really is a place to locate forget judgment.
This is what I want to answer. Anyway , also learned a lot of things, thank you ! Results posted
------ For reference only ------------------------------------ ---
and learn ! ! ! Operating system white school !
没有评论:
发表评论