The MSDtsSrvr.exe (SSIS service) process' memory working set size continues to grow until it consumes the memory on the machine.
Some of the packages do use a custom component. Could this be responsible for leaking memory in the MSDtsSrvr.exe process? Has a memory leak been confirmed in this process for the "first-party" components anyway?
More information about the possible memory leak.
The packages are run in in a CLR process with calls Package.Execute(), rather than being executed via dtexec or dtexecui or the designer. The reason for this approach is the ability to write a custom event handler with access to the package pointer, which other log handlers cannot do. The process runs Package.Execute() and exits, so there are 10,000 short-lived processes, each of which call Package.Execute() once.
To be slightly more specific about what the packages are doing, each of the 10,000 packages generally correlates to 1 source file, loading in dummy (aka inferred) dimension records, and then populating a fact table via a surrogate key pipeline.
No comments:
Post a Comment