1. using a CSV download.
2. transfer large JVM, Tomcat memory .
current customers do not agree with these two methods , you have any way to do this ?
------ Solution ---------------------------------------- ----
memory overflow because the load caused by too much data .
If so, you can load up data separately , and then were exported to multiple forms or multiple files .
That press paging query for export.
------ Solution ---------------------------------------- ----
Excel 2010 and Excel 2007 ( ie xlsx format ) worksheet size is 16,384 columns × 1,048,576 rows .
do not know can not meet the requirements , you can not , then you txt save , but the file may be too large, open slower
------ Solution -------------- ------------------------------
That should not be difficult . You can load it every time an Excel file and loads the first Sheet, paged data loaded every time the last line from the Sheet start adding , so OK .
------ Solution ---------------------------------------- ----
I think it is because of memory overflow loading too many Model, rather than a result of loading Excel files .
an Excel document just how much ah. I once had a load about 3G movie files are no problem.
------ Solution ---------------------------------------- ----
Yes, you should determine the Java memory leaks reported , not reported in the database memory overflow , Oracle loaded data may also be reported if more memory overflow .
If the Java memory leaks reported , then according to my way of thinking should be able to solve .
you're using a framework like Hibernate , check out a List of it . List is encapsulated inside the store Java objects , is that right ?
------ Solution ---------------------------------------- ----
management hibernate caching mechanism , hibernate 's session has flush, clear and other methods , allowing a smaller memory consumption
------ Solution ---------- ----------------------------------
can generate multiple temporary excel file . . Then combined into an excel file. . Write a combined class . . .
------ Solution ---------------------------------------- ----
brother have encountered this problem , customers have to turn out all the data , and sometimes large amount of data , even more than the maximum number of rows EXCEL really perverted , then the idea is : EXCEL file into multiple small , then use the method of the Java class libraries compressed into a ZIP file export XXX.zip download .
------ Solution ---------------------------------------- ----
I have studied this issue , ultimately generate multiple excel file to play zip package , too. Memory overflow is due to the amount of data is too big, that white is because too many objects created cell , leading to memory overflow , I could not find a better solution , jxl can also export excel, but will not overflow , but excel2003 maximum number of rows is 65535 , the amount of data if you spill it, or need to generate 2007 .
------ Solution ---------------------------------------- ----
I do not know the landlord excel database does not support export data bits , if supported , you can talk about the data into a temporary table , and then exported. This method is a little rotten .
------ For reference only ---------------------------------------
end-user requirements a sheet to accommodate all data , how to integrate the previously exported multiple sheet?
------ For reference only -------------------------------------- -
Brother, read before write , documents become increasingly large , POI or read the contents of the previous go , and finally overflow.
I tried the next 20,000 lines also, thirty thousand can not. My goal is 60,000 , you have a way to do this ?
------ For reference only -------------------------------------- -
or you do not encapsulate the data into Model, direct use of jdbc ResultSet to fill in the blank Excel should also not out of memory .
------ For reference only -------------------------------------- -
This sentence can extend it, do not quite understand .
Further data Well , I was taken out to put list, let alone sixty thousand , one hundred thousand lines not reported overflow error . Exceptions are made in an Excel file using POI generated .
------ For reference only ---------------------------------- -----
is now up to twenty thousand lines .
------ For reference only -------------------------------------- -
upstairs two , I'm already using simulated data , do not go backstage . 28000 but still could not pass this barrier, not to mention the 60,000 it. Data generation process never reported abnormal , java.lang.OutOfMemoryError: Java heap space when all is out in the generated Excel .
I tried it with a loop directly generated , segmented generate constantly read and write files in two ways, always return java.lang.OutOfMemoryError: Java heap space this exception.
------ For reference only -------------------------------------- -
I first refer to the online program, but he is based POI3.1 , and to my POI3.5 API is not available in a lot of it.
As mentioned above, I used a stepwise manner , the generated temporary files , and then merge the target file will be found more and more, eventually led to the overflow.
------ For reference only -------------------------------------- -
Brother, so dry the customer is not agree.
They need such a solution :
1. an XLS file downloads.
2. If the data is less than 60,000 lines , only made a Sheet.
3. greater than 60,000 lines to 60,000 units of Sheet storage.
Unfortunately, little brother is still hovering around only 26,000 rows .
------ For reference only -------------------------------------- -
or landlord can construct a file named xxx.xls, and then write the contents
| 1 | 2 | 3 | 4 |
| 1 | 2 | 3 | 4 |
| 1 | 2 | 3 | 4 |
| 1 | 2 | 3 | 4 |
do not know what they do with this excel data also subsequent operation is not , if only to see , so can it.
------ For reference only -------------------------------------- -
there is a read operation is to come , but also with POI. Halo ah.
------ For reference only ---------------------------------- -----
just some time ago I have studied this issue , asked some people, but whether it is poi or jxl, seemingly have no way to solve
excel no matter how prepared are unified operation last load into memory first written into the final , this time the amount of data will be ringing off the hook .
Only a disguised solution is full of how much data to regenerate a temporary excel finally merge, but I do not know how to achieve
------ For reference only -------- -------------------------------
CSV bar. .
disagree let them engage in their own .
------ For reference only -------------------------------------- -
I have encountered this problem , poi generated excel document does have a data volume issue, but also about the 28000 line data, more than a certain amount of data , there must be such a problem.
solution is then changed to csv, else no way.
------ For reference only ---------------------------------- -----
1. an XLS file downloads.
2. If the data is less than 60,000 lines , only made a Sheet.
3. greater than 60,000 lines to 60,000 units of Sheet storage.
that data is not read into the list, directly into the cell where it just generates a bit long time nothing more.
10W of the data probably more than 20 S, on the use of poi.
------ For reference only -------------------------------------- -
28000 and I tested the ceiling almost.
------ For reference only -------------------------------------- -
check out the database directly put cell, but also to accommodate the overflow error instead of 10W ? Too, to look at the code chant.
------ For reference only -------------------------------------- -
I really have not tried this method , the landlord try to send a conclusion out .
I do not know how to engage Microsoft's own landlord look of wood with c language export excel example , if anything, can be encapsulated into a dll using jni call to try.
------ For reference only -------------------------------------- -
may very soil , but it does not overflow , jointly improve it
a cell a cell of the new Yeah, why would spill it ?
import java.sql.ResultSet;
import java.sql.SQLException;
import org.apache.poi.hssf.usermodel.HSSFCell;
import org.apache.poi.hssf.usermodel.HSSFRow;
import org.apache.poi.hssf.usermodel.HSSFSheet;
import org.apache.poi.hssf.usermodel.HSSFWorkbook;
public class infoExportExcel {
/**
*
* @param intRowCount 总行数
* @param sheetCount 一个sheet表内的行数
* @param wb excel
* @param sheetname sheet表名
* @param columNumber 列数
* @param tableHeader 表头名
* @param rs 查询结果集
* @throws SQLException
*/
public infoExportExcel(int intRowCount, int sheetCount, HSSFWorkbook wb, String sheetname, int columNumber, String[] tableHeader, ResultSet rs) throws SQLException{
int allSheetCount = (intRowCount + sheetCount -1)/sheetCount; //计算总工作表数,分表
if(allSheetCount > 1){ //分多个sheet表来显示
int q = 0;//sheet数
while(q < allSheetCount ){
HSSFSheet sheet = wb.createSheet(sheetname + q); //sheet表名依次为表0、表1、表2...
HSSFRow rowHeader = null;//表头
rowHeader = sheet.createRow(0);
for(int u = 0; u < columNumber; u++){
HSSFCell headerCell = rowHeader.createCell(u);
headerCell.setCellValue(tableHeader[u]);
}
int p = 1; //行
HSSFRow rowData = null;//表内容
while(rs.next() && p < sheetCount){
rowData = sheet.createRow(p);
HSSFCell cellData = null;
for(int j = 0; j < columNumber; j++){
cellData = rowData.createCell(j);
cellData.setCellValue(rs.getString(j + 1));
}
p++;
if(p > sheetCount){//大于行数后，下一个sheet表从1继续开始
p = 1;
}
}
q++;
}
}else{
HSSFSheet sheet = wb.createSheet(sheetname);
HSSFRow rowHeader = null;
rowHeader = sheet.createRow(0);
for(int u = 0; u < columNumber; u++){
HSSFCell headerCell = rowHeader.createCell(u);
headerCell.setCellValue(tableHeader[u]);
}
int p = 1;
HSSFRow rowData = null;
while(rs.next() && p < (intRowCount + 1)){
rowData = sheet.createRow(p);
HSSFCell cellData = null;
for(int j = 0; j < columNumber; j++){
cellData = rowData.createCell(j);
cellData.setCellValue(rs.getString(j + 1));
}
p++;
}
}
}
}
------ For reference only ----------------------------------- ----
upstairs do not understand why the code does not overflow , is it takes time to release memory POI ?
In addition, I found reading a 5W multi-line XLS is no problem , the problem may not lie with the reading here .
------ For reference only -------------------------------------- -
Will you be able to put up a Sheet how many lines ?
------ For reference only -------------------------------------- -
you do not say what export excel Download problems ? How they read the == ~ ~
Where do you think the problem ? This code is no problem here in my country, written before the time thought so much of , subsequently sought points table shows when it added a judge, you can export correctly
------ For reference only - --------------------------------------
time trial yesterday, just 5q 1W and 5W, can
------ For reference only ----------------------------- ----------
TO lxwankkk:
1. Could you set up the server memory size yet ?
2. >> Did not you say export excel download problems it ? How they read the == ~ ~
that I was going to test read a large file will not overflow phenomenon .
3. >> Where do you think the problem ? This code is no problem here in my country, written before the time thought so much of , subsequently sought points table shows when it added a judge, you can export correctly
problem here , and can not understand why the first read a List and then output the read XLS be wrong to write directly after recording no problem .
------ For reference only -------------------------------------- -
1, no
3, ah , years ago, seems to have discussed this issue with the people , he is also the first place in the list , and then will report overflow. If that is 21 floor bar.
ah , may still be where it is limited .
------ For reference only -------------------------------------- -
I generate clean out after the end of the list found in the amount of data on the 30,000 , more than not tried .
so will I try to obtain records on the output of the case .
------ For reference only -------------------------------------- -
just tried to obtain records on the output mode , the amount of data can be increased to 35,000 , but still can not get 4W .
code is as follows :
Servlet code :
public class DownloadServlet extends HttpServlet {
private static final long serialVersionUID = 56890894234786L;
public void doPost(HttpServletRequest request, HttpServletResponse response)
throws ServletException, java.io.IOException {
String fileName="download.xls";
response.setHeader("Content-disposition", "attachment; filename="+fileName);// 设定输出文件头
response.setContentType("application/msexcel");// 定义输出类型
try{
int rowCount=Integer.parseInt(request.getParameter("rowCount"));
// 表头行
String[] headers=new String[]{"更新ID","账期","基站编号","基站名称","站点状态","部门名称","站点类型","占用类型","预提(元)","未核销金额","上期未核销","开始月份","结束月份","上期抄表数","本期抄表数","电价","电量","本期报账(元)","补提(元)","预提汇总(元)","成本中心","专业","本期报账单号","基站类别","线损"};
DownloadService service=new DownloadService();
HSSFWorkbook workbook=service.generateWorkbook(rowCount, headers.length);
// 输出部分
OutputStream os = response.getOutputStream();// 取得输出流
workbook.write(os);
os.flush();
os.close();
}catch(Exception ex){
ex.printStackTrace();
}
return ;
}
public void doGet(HttpServletRequest request, HttpServletResponse response)
throws ServletException, java.io.IOException {
doPost(request, response);
}
}
Service Code:
public class DownloadService{
/**
* 生成工作簿对象
* @param rowCount
* @param columnCount
* @return
*/
public HSSFWorkbook generateWorkbook(int rowCount,int columnCount){
HSSFWorkbook workbook = new HSSFWorkbook(); //产生工作簿对象
HSSFSheet sheet = workbook.createSheet(); //产生工作表对象
String value=null;
for(int i=0;i<rowCount;i++){
HSSFRow row = sheet.createRow(i);//创建一行
for(int j=0;j<columnCount;j++){
value=""+i+","+j;
HSSFCell cell = row.createCell(j);
cell.setCellValue(value);
cell=null;
}
}
return workbook;
}
}
------ For reference only ---------------------------------- -----
has a relationship with the input and output streams it , I use this
ServletOutputStream out = response.getOutputStream();
BufferedOutputStream bos = new BufferedOutputStream(out);
wb.write(bos);
------ For reference only ----------------------------------- ----
also tried it, seems to be no relationship .
------ For reference only -------------------------------------- -
now or in the output between 35000-36000 .
------ For reference only -------------------------------------- -
== ~ ~ then you continue to study under the bar. . . Fruitful sharing under 0.0
------ For reference only --------------------------------- ------
this error , such as maggots tarsal general.
java.lang.OutOfMemoryError: Java heap space
at org.apache.poi.poifs.storage.DocumentBlock.
at org.apache.poi.poifs.storage.DocumentBlock.
at org.apache.poi.poifs.filesystem.POIFSDocument.
at org.apache.poi.poifs.filesystem.POIFSDocument.
at org.apache.poi.poifs.filesystem.DirectoryNode.createDocument (DirectoryNode.java: 309)
at org.apache.poi.poifs.filesystem.POIFSFileSystem.createDocument (POIFSFileSystem.java: 265)
at org.apache.poi.hssf.usermodel.HSSFWorkbook.write (HSSFWorkbook.java: 1157)
at com.heyang.action.DownloadServlet.doPost (DownloadServlet.java: 47)
at com.heyang.action.DownloadServlet.doGet (DownloadServlet.java: 60)
at javax.servlet.http.HttpServlet.service (HttpServlet.java: 690)
at javax.servlet.http.HttpServlet.service (HttpServlet.java: 803)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter (ApplicationFilterChain.java: 290)
at org.apache.catalina.core.ApplicationFilterChain.doFilter (ApplicationFilterChain.java: 206)
at org.apache.catalina.core.StandardWrapperValve.invoke (StandardWrapperValve.java: 228)
at org.apache.catalina.core.StandardContextValve.invoke (StandardContextValve.java: 175)
at org.apache.catalina.core.StandardHostValve.invoke (StandardHostValve.java: 128)
at org.apache.catalina.valves.ErrorReportValve.invoke (ErrorReportValve.java: 104)
at org.apache.catalina.core.StandardEngineValve.invoke (StandardEngineValve.java: 109)
at org.apache.catalina.connector.CoyoteAdapter.service (CoyoteAdapter.java: 216)
at org.apache.coyote.http11.Http11Processor.process (Http11Processor.java: 844)
at org.apache.coyote.http11.Http11Protocol $ Http11ConnectionHandler.process (Http11Protocol.java: 634)
at org.apache.tomcat.util.net.JIoEndpoint $ Worker.run (JIoEndpoint.java: 445)
at java.lang.Thread.run (Unknown Source)
------ For reference only ---------------------------------- -----
=. =
other experts bar Oh , I've no problem yo
------ For reference only ------------------------ ---------------
Oh, I have encountered .
my side of the practice is to excel compressed into a zip, and then download .
because of the particularity of the contents of my side excel after compression can reduce the very large space
------ For reference only ---------------- -----------------------
This can be a trial , I wrote a
------ For reference only ---------------------- -----------------
may be able to try this , you read 30,000 when the line is disconnected , the then re- read the output ! Like ODBC connection will give an error if more than 300
------ For reference only ---------------------------- -----------
ask you how to solve this problem , and now I have a similar problem , in more than 28,000 rows will be such a problem !
------ For reference only -------------------------------------- -
Bucai ah , I also encountered this problem , about 10W in the EXCEL records into the database, an error ah.
java.lang.OutOfMemoryError: Java heap space
everyone studies , how to solve it ?
landlord , now solved?
------ For reference only -------------------------------------- -
I have encountered such a problem , but only on the local machine memory overflow occurs ,
If the application is deployed to the server, because the server for performance reasons , basically to fill
meet customer demand, there have been no memory overflow temporary phenomenon , so I suggest you go to the server to try (
open big JVM memory can also try , quietly , do not give him say , ha ha ) .
If it is not, I would recommend using cvs download ( customer does not agree with it, such suffixes
file name , the default is to use excel basically opened , basically it will not have any effect )
------ For reference only ----------- ----------------------------
I have encountered such a problem , but only on the local machine memory overflow occurs case ,
If the application is deployed to the server, because the server for performance reasons , basically to fill
meet customer demand, there have been no memory overflow temporary phenomenon , so I suggest you go to the server to try (
open big JVM memory can also try , quietly , do not give him say , ha ha ) .
If it is not, I would recommend using cvs download ( customer does not agree with it, such suffixes
file name , the default is to use excel basically opened , basically it will not have any effect )
------ For reference only ----------- ----------------------------
you can discuss how to squeeze half not discussed ? How to solve ah , pro, I also encountered this problem. . . Sub files can be combined or when ringing off the hook . . I use JXL. ------ For reference only -------------------------------------- -
03 , then use eventusermodel mode
07 , then use the stream output mode
------ For reference only --------------------------- ------------
reference java to excel export large amounts of data
------ For reference only ----- ----------------------------------
Well, I have encountered, but would like a stupid way can to an Excel file for each sheet tab 100W write data , an ordinary computer is no problem.
I use the POI.
using the POI SXSSFWorkbook objects , such as:
SXSSFWorkbook workbook = new SXSSFWorkbook (100);
The 100 is the number of cache lines , so write to Excel , write a 1 million is not a problem , very fast .
If it is a need to re- write an existing Excel file data and existing Excel file has a lot of data , such as the one million .
written directly to the file will still overflow , it is best to write each new Excel file and then call VB, execute two copies of Excel files between the sheet , so that can be successful
------ For reference only ---------------------------------- -----
today have encountered this problem. . .
After reading a bit back great experience , finished last thought every line of data is removed from the list of this data .
This would solve the memory situation , we can try ~ ~
------ For reference only ------------------- --------------------
just can . . Now wrong. . Hey . . Failed. .
------ For reference only -------------------------------------- -
just can . . Now wrong. . Hey . . Failed. .
53 floor, said method can be used
没有评论:
发表评论