http://www.sqlslayer.com/wp/2009/10/13/memory-leak-in-for-loop-container/ Making SQL do what we want it to do. Sat, 06 Aug 2011 03:28:56 -0700 http://wordpress.org/?v=2.8.4 hourly 1 http://www.sqlslayer.com/wp/2009/10/13/memory-leak-in-for-loop-container/comment-page-1/#comment-9 TheSQLGeek Thu, 15 Oct 2009 00:34:13 +0000 http://www.sqlslayer.com/wp/?p=129#comment-9 I wanted to provide some quick information on the post above. SSIS uses buffers to manage the work that it handles in memory. You will not find these documented anywhere. You just have to know the person(s) who wrote that piece in SSIS. In SQL Server 2005 the buffers used inside of SSIS are as follows; Physical, Prime, Column View, Virtual. Now depending on how the task blocking or non blocking it utilizes the buffers in different ways. One thing that is important is the sheer size of the graph with SSIS. (Graph is the official term for the work/data flow orchestrations in SSIS) For performance and memory reasons to perf tune your package you need to size your graph to your buffer size that you setup in SSIS. Engine threads and work/transformation types will cause this number to move around a bit. You may also want to setup up some disk buffer on your laptop if you need to run the package on the 1 GB RAM system. By the sounds of the task we are holding multiple datasets in the prime buffers and the column view buffer as well. This will cause RAM swelling. As the data is kept in more than one buffer. I hope that my explain is sorta clear. The most important thing in SSIS is sizing your graph to your buffer. You can always pre stage your data in memory (included in SQL 2008 as Source and Destination Buffers) with a small amount of effort as well. I wanted to provide some quick information on the post above. SSIS uses buffers to manage the work that it handles in memory. You will not find these documented anywhere. You just have to know the person(s) who wrote that piece in SSIS. In SQL Server 2005 the buffers used inside of SSIS are as follows; Physical, Prime, Column View, Virtual. Now depending on how the task blocking or non blocking it utilizes the buffers in different ways. One thing that is important is the sheer size of the graph with SSIS. (Graph is the official term for the work/data flow orchestrations in SSIS) For performance and memory reasons to perf tune your package you need to size your graph to your buffer size that you setup in SSIS. Engine threads and work/transformation types will cause this number to move around a bit. You may also want to setup up some disk buffer on your laptop if you need to run the package on the 1 GB RAM system. By the sounds of the task we are holding multiple datasets in the prime buffers and the column view buffer as well. This will cause RAM swelling. As the data is kept in more than one buffer. I hope that my explain is sorta clear. The most important thing in SSIS is sizing your graph to your buffer. You can always pre stage your data in memory (included in SQL 2008 as Source and Destination Buffers) with a small amount of effort as well.

]]>