Excess memory usage growth for a process

Discussion:

Eddie Lascu

2009-07-27 15:32:35 UTC

Hi guys,

I need your help with an issue I am having right now. I have several windows
services deployed. These services gather some XML files from the internet
and then parse the content and save the information in the database. The
database is Oracle 11g, so the access is done through ODP.NET. I had some
issues before (and some of you with good memory may recall my previous
questions) where I noticed the memory used by these services growing very
high. At the time, I understood that some classes in the ODP.NET library use
some JAVA code and they are known for not cleaning properly after them. As a
result I rewrote my code in such a way that every dynamic allocation of an
object from the ODP.NET library was encompassed in an "using" statement,
like this:

using (OracleCommand objDbCommand = new OracleCommand(strSqlSelectStatement,
objConnection))

All was fine for a very long time. All my services kept a constant size when
it came to the total memory used, a size that hovered around 55-60 MB. This
was the case for months. At the beginning I checked the size almost on a
daily basis. After a while, my confidence grew and I only checked it once a
week or so. Every time I looked at the size of all my processes (I had 4
that followed pretty much the same approach on dealing with those Oracle
objects) the size was around that 55-60 MB mark.

Now, all of a sudden, this last weekend, the size for some of the services
shut up to about 690 MB, despite the fact the service was running with the
exact same code as it did a month ago. This represents a potential fatal
problem for the server that can get to a point where it can no longer
allocate memory, so I need to find a way to understand what the heck is
going on. Unfortunately, I do not have any metrics implemented in my
application to log the total amount of memory used. That would have been a
very good indication on when exactly did this surge started to happen. I
could have checked the Event Viewer to see if the server got hit by a
nuclear bomb or something on that precise moment. I really don't get this. I
mean that's why we moved to .NET in the first place, to be rid of such
concerns, like making sure all dynamically allocated objects are then
destroyed.

Can anyone give me an idea on how to investigate this issue? Are there tools
out there that could be used to pick at the memory allocated by a process
and see what are those objects that take up so much memory? For the time
being, I have the processes running, but two of them are at or close to 690
MB and I don't know how long will I be able to keep them running.

If you had an issue like this, what would you try to do?

Any suggestion will be greatly appreciated.

TIA,

Eddie

===================================
View archives and manage your subscription(s) at http://peach.ease.lsoft.com/archives

John Warner

2009-07-27 15:49:03 UTC

Permalink

Not what you are seeking, but has anything on the server changed? An
Oracle patch or something of that nature?

John Warner

-----Original Message-----
From: Discussion of advanced .NET topics. [mailto:ADVANCED-
Sent: Monday, July 27, 2009 11:33 AM
Subject: [ADVANCED-DOTNET] Excess memory usage growth for a process
Hi guys,
I need your help with an issue I am having right now. I have several windows
services deployed. These services gather some XML files from the internet
and then parse the content and save the information in the database. The
database is Oracle 11g, so the access is done through ODP.NET. I had some
issues before (and some of you with good memory may recall my previous
questions) where I noticed the memory used by these services growing very
high. At the time, I understood that some classes in the ODP.NET library use
some JAVA code and they are known for not cleaning properly after them. As a
result I rewrote my code in such a way that every dynamic allocation of an
object from the ODP.NET library was encompassed in an "using" statement,
using (OracleCommand objDbCommand = new
OracleCommand(strSqlSelectStatement,
objConnection))
All was fine for a very long time. All my services kept a constant size when
it came to the total memory used, a size that hovered around 55-60 MB. This
was the case for months. At the beginning I checked the size almost on a
daily basis. After a while, my confidence grew and I only checked it once a
week or so. Every time I looked at the size of all my processes (I had 4
that followed pretty much the same approach on dealing with those Oracle
objects) the size was around that 55-60 MB mark.
Now, all of a sudden, this last weekend, the size for some of the services
shut up to about 690 MB, despite the fact the service was running with the
exact same code as it did a month ago. This represents a potential fatal
problem for the server that can get to a point where it can no longer
allocate memory, so I need to find a way to understand what the heck is
going on. Unfortunately, I do not have any metrics implemented in my
application to log the total amount of memory used. That would have been a
very good indication on when exactly did this surge started to happen. I
could have checked the Event Viewer to see if the server got hit by a
nuclear bomb or something on that precise moment. I really don't get this. I
mean that's why we moved to .NET in the first place, to be rid of such
concerns, like making sure all dynamically allocated objects are then
destroyed.
Can anyone give me an idea on how to investigate this issue? Are there tools
out there that could be used to pick at the memory allocated by a process
and see what are those objects that take up so much memory? For the time
being, I have the processes running, but two of them are at or close to 690
MB and I don't know how long will I be able to keep them running.
If you had an issue like this, what would you try to do?
Any suggestion will be greatly appreciated.
TIA,
Eddie
===================================
View archives and manage your subscription(s) at
http://peach.ease.lsoft.com/archives

===================================
View archives and manage your subscription(s) at http://peach.ease.lsoft.com/archives

Eddie Lascu

2009-07-27 16:00:08 UTC

Permalink

Hi John,

I could check that with our system administrators. Do you think that can be
a possible explanation?
I just talked with one of the DB Admin and she said she made some changes on
some Stored Produces, but nothing as dramatic as a patch update. She also
said that the drivers for the network cards were updated last week, but I
can't see how that can affect the memory used by my processes.

Thanks,
Eddie

-----Original Message-----
From: Discussion of advanced .NET topics.
[mailto:ADVANCED-***@PEACH.EASE.LSOFT.COM] On Behalf Of John Warner
Sent: Monday, July 27, 2009 11:49 AM
To: ADVANCED-***@PEACH.EASE.LSOFT.COM
Subject: Re: [ADVANCED-DOTNET] Excess memory usage growth for a process

Not what you are seeking, but has anything on the server changed? An
Oracle patch or something of that nature?

John Warner

John Warner

2009-07-27 16:28:13 UTC

Permalink

I don't disagree with you just note that before 'some change' working OK
no memory leaks, after change, memory leak. We both know your code did not
recompile itself to include a memory leak ...
That reminds has the framework/runtimes been updated/patched?

John Warner

-----Original Message-----
From: Discussion of advanced .NET topics. [mailto:ADVANCED-
Sent: Monday, July 27, 2009 12:00 PM
Subject: Re: [ADVANCED-DOTNET] Excess memory usage growth for a process
Hi John,
I could check that with our system administrators. Do you think that can be
a possible explanation?
I just talked with one of the DB Admin and she said she made some

changes

on
some Stored Produces, but nothing as dramatic as a patch update. She also
said that the drivers for the network cards were updated last week, but I
can't see how that can affect the memory used by my processes.
Thanks,
Eddie
-----Original Message-----
From: Discussion of advanced .NET topics.
John Warner
Sent: Monday, July 27, 2009 11:49 AM
Subject: Re: [ADVANCED-DOTNET] Excess memory usage growth for a process
Not what you are seeking, but has anything on the server changed? An
Oracle patch or something of that nature?
John Warner

-----Original Message-----
From: Discussion of advanced .NET topics. [mailto:ADVANCED-
Sent: Monday, July 27, 2009 11:33 AM
Subject: [ADVANCED-DOTNET] Excess memory usage growth for a

process

Hi guys,
I need your help with an issue I am having right now. I have several

windows

services deployed. These services gather some XML files from the

internet

and then parse the content and save the information in the database. The
database is Oracle 11g, so the access is done through ODP.NET. I had

some

issues before (and some of you with good memory may recall my previous
questions) where I noticed the memory used by these services growing

very

high. At the time, I understood that some classes in the ODP.NET

library

use

some JAVA code and they are known for not cleaning properly after

them.

As a

result I rewrote my code in such a way that every dynamic allocation

object from the ODP.NET library was encompassed in an "using" statement,
using (OracleCommand objDbCommand = new
OracleCommand(strSqlSelectStatement,
objConnection))
All was fine for a very long time. All my services kept a constant

size

when

it came to the total memory used, a size that hovered around 55-60 MB.

This

was the case for months. At the beginning I checked the size almost on a
daily basis. After a while, my confidence grew and I only checked it

once a

week or so. Every time I looked at the size of all my processes (I had 4
that followed pretty much the same approach on dealing with those Oracle
objects) the size was around that 55-60 MB mark.
Now, all of a sudden, this last weekend, the size for some of the

services

shut up to about 690 MB, despite the fact the service was running with

the

exact same code as it did a month ago. This represents a potential fatal
problem for the server that can get to a point where it can no longer
allocate memory, so I need to find a way to understand what the heck is
going on. Unfortunately, I do not have any metrics implemented in my
application to log the total amount of memory used. That would have

been

very good indication on when exactly did this surge started to happen. I
could have checked the Event Viewer to see if the server got hit by a
nuclear bomb or something on that precise moment. I really don't get

this. I

mean that's why we moved to .NET in the first place, to be rid of such
concerns, like making sure all dynamically allocated objects are then
destroyed.
Can anyone give me an idea on how to investigate this issue? Are there

tools

out there that could be used to pick at the memory allocated by a

process

and see what are those objects that take up so much memory? For the

time

being, I have the processes running, but two of them are at or close

690

MB and I don't know how long will I be able to keep them running.
If you had an issue like this, what would you try to do?
Any suggestion will be greatly appreciated.
TIA,
Eddie
===================================
View archives and manage your subscription(s) at
http://peach.ease.lsoft.com/archives

===================================
View archives and manage your subscription(s) at http://peach.ease.lsoft.com/archives

Shawn Wildermuth

2009-07-27 16:49:02 UTC

Permalink

Also, your 'using' statement stuff should help, but are you sure that all
your object (XML classes and db connection classes) that support IDisposable
are being cleaned up?

Thanks,

Shawn Wildermuth
http://wildermuth.com
https://agilitrain.com
Microsoft MVP (C#), MCSD.NET, Author and Speaker

The Silverlight Tour is coming to a city near you!

-----Original Message-----
From: Discussion of advanced .NET topics.
[mailto:ADVANCED-***@PEACH.EASE.LSOFT.COM] On Behalf Of John Warner
Sent: Monday, July 27, 2009 12:28 PM
To: ADVANCED-***@PEACH.EASE.LSOFT.COM
Subject: Re: [ADVANCED-DOTNET] Excess memory usage growth for a process

I don't disagree with you just note that before 'some change' working OK
no memory leaks, after change, memory leak. We both know your code did not
recompile itself to include a memory leak ...
That reminds has the framework/runtimes been updated/patched?

John Warner

changes

-----Original Message-----
From: Discussion of advanced .NET topics. [mailto:ADVANCED-
Sent: Monday, July 27, 2009 11:33 AM
Subject: [ADVANCED-DOTNET] Excess memory usage growth for a

process

Hi guys,
I need your help with an issue I am having right now. I have several

windows

services deployed. These services gather some XML files from the

internet

and then parse the content and save the information in the database. The
database is Oracle 11g, so the access is done through ODP.NET. I had

some

issues before (and some of you with good memory may recall my previous
questions) where I noticed the memory used by these services growing

very

high. At the time, I understood that some classes in the ODP.NET

library

use

some JAVA code and they are known for not cleaning properly after

them.

As a

result I rewrote my code in such a way that every dynamic allocation

size

when

it came to the total memory used, a size that hovered around 55-60 MB.

This

was the case for months. At the beginning I checked the size almost on a
daily basis. After a while, my confidence grew and I only checked it

once a

services

shut up to about 690 MB, despite the fact the service was running with

the

been

this. I

tools

out there that could be used to pick at the memory allocated by a

process

and see what are those objects that take up so much memory? For the

time

being, I have the processes running, but two of them are at or close

690

Eddie Lascu

2009-07-29 16:33:18 UTC

Permalink

Hi guys,

In light of this article:

http://www.simple-talk.com/dotnet/.net-framework/the-dangers-of-the-large-object-heap/

I am no longer sure what I have is a memory leak. I could see how some of the things that happen in my application match the scenarios described in the above article. Hence, I am ready to think about a workaround. My services are set up to run automatically and restart in case they crash. While they never crash, I could leverage that feature and use it to fix this excess memory usage that I experience every once in a blue moon. What I thought was to monitor the memory used and terminate the application in case it exceeds a set threshold. The Windows Service Manager will then restart the service and all the memory will be reclaimed by the OS.

My question at this point is whether the .NET Framework has some metric classes that can help monitor the total amount of memory allocated within a process. Surely System.Diagnostics must have something like that. Is anyone aware of a way to find out what memory is used by a process, just like the Task Manager does?

Cheers,
Eddie

-----Original Message-----
From: Discussion of advanced .NET topics. [mailto:ADVANCED-***@PEACH.EASE.LSOFT.COM] On Behalf Of Shawn Wildermuth
Sent: Monday, July 27, 2009 12:49 PM
To: ADVANCED-***@PEACH.EASE.LSOFT.COM
Subject: Re: [ADVANCED-DOTNET] Excess memory usage growth for a process

Also, your 'using' statement stuff should help, but are you sure that all
your object (XML classes and db connection classes) that support IDisposable
are being cleaned up?

Thanks,

Shawn Wildermuth
http://wildermuth.com
https://agilitrain.com
Microsoft MVP (C#), MCSD.NET, Author and Speaker

The Silverlight Tour is coming to a city near you!

-----Original Message-----
From: Discussion of advanced .NET topics.
[mailto:ADVANCED-***@PEACH.EASE.LSOFT.COM] On Behalf Of John Warner
Sent: Monday, July 27, 2009 12:28 PM
To: ADVANCED-***@PEACH.EASE.LSOFT.COM
Subject: Re: [ADVANCED-DOTNET] Excess memory usage growth for a process

I don't disagree with you just note that before 'some change' working OK
no memory leaks, after change, memory leak. We both know your code did not
recompile itself to include a memory leak ...
That reminds has the framework/runtimes been updated/patched?

John Warner

changes

-----Original Message-----
From: Discussion of advanced .NET topics. [mailto:ADVANCED-
Sent: Monday, July 27, 2009 11:33 AM
Subject: [ADVANCED-DOTNET] Excess memory usage growth for a

process

Hi guys,
I need your help with an issue I am having right now. I have several

windows

services deployed. These services gather some XML files from the

internet

and then parse the content and save the information in the database. The
database is Oracle 11g, so the access is done through ODP.NET. I had

some

issues before (and some of you with good memory may recall my previous
questions) where I noticed the memory used by these services growing

very

high. At the time, I understood that some classes in the ODP.NET

library

use

some JAVA code and they are known for not cleaning properly after

them.

As a

result I rewrote my code in such a way that every dynamic allocation

size

when

it came to the total memory used, a size that hovered around 55-60 MB.

This

was the case for months. At the beginning I checked the size almost on a
daily basis. After a while, my confidence grew and I only checked it

once a

services

shut up to about 690 MB, despite the fact the service was running with

the

been

this. I

tools

out there that could be used to pick at the memory allocated by a

process

and see what are those objects that take up so much memory? For the

time

being, I have the processes running, but two of them are at or close

690

===================================
View archives and manage your subscription(s) at
http://peach.ease.lsoft.com/archives

===================================
View archives and manage your subscription(s) at http://peach.ease.lsoft.com/archives

===================================
View archives and manage your subscription(s) at http://peach.ease.lsoft.com/archives

Greg Young

2009-07-29 16:38:23 UTC

Permalink

Task manager shows you working set not total memory used but yes the
process class can help you with that (Process.WorkingSet as an
example).

Have you verified the LOH being the problem by using SOS?

Cheers,

Greg

Post by Eddie Lascu
Hi guys,
http://www.simple-talk.com/dotnet/.net-framework/the-dangers-of-the-large-object-heap/
I am no longer sure what I have is a memory leak. I could see how some of the things that happen in my application match the scenarios described in the above article. Hence, I am ready to think about a workaround. My services are set up to run automatically and restart in case they crash. While they never crash, I could leverage that feature and use it to fix this excess memory usage that I experience every once in a blue moon. What I thought was to monitor the memory used and terminate the application in case it exceeds a set threshold. The Windows Service Manager will then restart the service and all the memory will be reclaimed by the OS.
My question at this point is whether the .NET Framework has some metric classes that can help monitor the total amount of memory allocated within a process. Surely System.Diagnostics must have something like that. Is anyone aware of a way to find out what memory is used by a process, just like the Task Manager does?
Cheers,
Eddie
-----Original Message-----
Sent: Monday, July 27, 2009 12:49 PM
Subject: Re: [ADVANCED-DOTNET] Excess memory usage growth for a process
Also, your 'using' statement stuff should help, but are you sure that all
your object (XML classes and db connection classes) that support IDisposable
are being cleaned up?
Thanks,
Shawn Wildermuth
http://wildermuth.com
https://agilitrain.com
Microsoft MVP (C#), MCSD.NET, Author and Speaker
The Silverlight Tour is coming to a city near you!
-----Original Message-----
From: Discussion of advanced .NET topics.
Sent: Monday, July 27, 2009 12:28 PM
Subject: Re: [ADVANCED-DOTNET] Excess memory usage growth for a process
I don't disagree with you just note that before 'some change' working OK
no memory leaks, after change, memory leak. We both know your code did not
recompile itself to include a memory leak ...
That reminds has the framework/runtimes been updated/patched?
John Warner

a possible explanation?
I just talked with one of the DB Admin and she said she made some

changes

on
some Stored Produces, but nothing as dramatic as a patch update. She

also

said that the drivers for the network cards were updated last week, but

can't see how that can affect the memory used by my processes.
Thanks,
Eddie
-----Original Message-----
From: Discussion of advanced .NET topics.
John Warner
Sent: Monday, July 27, 2009 11:49 AM
Subject: Re: [ADVANCED-DOTNET] Excess memory usage growth for a process
Not what you are seeking, but has anything on the server changed? An
Oracle patch or something of that nature?
John Warner

-----Original Message-----
From: Discussion of advanced .NET topics. [mailto:ADVANCED-
Sent: Monday, July 27, 2009 11:33 AM
Subject: [ADVANCED-DOTNET] Excess memory usage growth for a

process

Hi guys,
I need your help with an issue I am having right now. I have several

windows

services deployed. These services gather some XML files from the

internet

and then parse the content and save the information in the database.

The

database is Oracle 11g, so the access is done through ODP.NET. I had

some

issues before (and some of you with good memory may recall my previous
questions) where I noticed the memory used by these services growing

very

high. At the time, I understood that some classes in the ODP.NET

library

use

some JAVA code and they are known for not cleaning properly after

them.

As a

result I rewrote my code in such a way that every dynamic allocation

object from the ODP.NET library was encompassed in an "using"

statement,

using (OracleCommand objDbCommand = new
OracleCommand(strSqlSelectStatement,
objConnection))
All was fine for a very long time. All my services kept a constant

size

when

it came to the total memory used, a size that hovered around 55-60 MB.

This

was the case for months. At the beginning I checked the size almost on

daily basis. After a while, my confidence grew and I only checked it

once a

week or so. Every time I looked at the size of all my processes (I had

that followed pretty much the same approach on dealing with those

Oracle

objects) the size was around that 55-60 MB mark.
Now, all of a sudden, this last weekend, the size for some of the

services

shut up to about 690 MB, despite the fact the service was running with

the

exact same code as it did a month ago. This represents a potential

fatal

problem for the server that can get to a point where it can no longer
allocate memory, so I need to find a way to understand what the heck

going on. Unfortunately, I do not have any metrics implemented in my
application to log the total amount of memory used. That would have

been

very good indication on when exactly did this surge started to happen.

could have checked the Event Viewer to see if the server got hit by a
nuclear bomb or something on that precise moment. I really don't get

this. I

tools

out there that could be used to pick at the memory allocated by a

process

and see what are those objects that take up so much memory? For the

time

being, I have the processes running, but two of them are at or close

690

===================================
View archives and manage your subscription(s) at
http://peach.ease.lsoft.com/archives
===================================
View archives and manage your subscription(s) at http://peach.ease.lsoft.com/archives
===================================
View archives and manage your subscription(s) at http://peach.ease.lsoft.com/archives

--
Les erreurs de grammaire et de syntaxe ont été incluses pour m'assurer
de votre attention

===================================
View archives and manage your subscription(s) at http://peach.ease.lsoft.com/archives

Eddie Lascu

2009-07-29 16:43:39 UTC

Permalink

I am using Reg Gate ANTS to profile the app, but it may take too much to
happen again. Before the spike I saw last weekend, the processes ran for 3-4
months with no significant change in the size of memory used.
Will look into Process.WorkingSet.

Thanks,
Eddie

-----Original Message-----
From: Discussion of advanced .NET topics.
[mailto:ADVANCED-***@PEACH.EASE.LSOFT.COM] On Behalf Of Greg Young
Sent: Wednesday, July 29, 2009 12:38 PM
To: ADVANCED-***@PEACH.EASE.LSOFT.COM
Subject: Re: [ADVANCED-DOTNET] Excess memory usage growth for a process

Task manager shows you working set not total memory used but yes the
process class can help you with that (Process.WorkingSet as an
example).

Have you verified the LOH being the problem by using SOS?

Cheers,

Greg

Post by Eddie Lascu
Hi guys,

http://www.simple-talk.com/dotnet/.net-framework/the-dangers-of-the-large-ob
ject-heap/

Post by Eddie Lascu
I am no longer sure what I have is a memory leak. I could see how some of

the things that happen in my application match the scenarios described in
the above article. Hence, I am ready to think about a workaround. My
services are set up to run automatically and restart in case they crash.
While they never crash, I could leverage that feature and use it to fix this
excess memory usage that I experience every once in a blue moon. What I
thought was to monitor the memory used and terminate the application in case
it exceeds a set threshold. The Windows Service Manager will then restart
the service and all the memory will be reclaimed by the OS.

Post by Eddie Lascu
My question at this point is whether the .NET Framework has some metric

classes that can help monitor the total amount of memory allocated within a
process. Surely System.Diagnostics must have something like that. Is anyone
aware of a way to find out what memory is used by a process, just like the
Task Manager does?

Post by Eddie Lascu
Cheers,
Eddie
-----Original Message-----
From: Discussion of advanced .NET topics.
Sent: Monday, July 27, 2009 12:49 PM
Subject: Re: [ADVANCED-DOTNET] Excess memory usage growth for a process
Also, your 'using' statement stuff should help, but are you sure that all
your object (XML classes and db connection classes) that support IDisposable
are being cleaned up?
Thanks,
Shawn Wildermuth
http://wildermuth.com
https://agilitrain.com
Microsoft MVP (C#), MCSD.NET, Author and Speaker
The Silverlight Tour is coming to a city near you!
-----Original Message-----
From: Discussion of advanced .NET topics.
Sent: Monday, July 27, 2009 12:28 PM
Subject: Re: [ADVANCED-DOTNET] Excess memory usage growth for a process
I don't disagree with you just note that before 'some change' working OK
no memory leaks, after change, memory leak. We both know your code did not
recompile itself to include a memory leak ...
That reminds has the framework/runtimes been updated/patched?
John Warner

a possible explanation?
I just talked with one of the DB Admin and she said she made some

changes

on
some Stored Produces, but nothing as dramatic as a patch update. She

also

said that the drivers for the network cards were updated last week, but

-----Original Message-----
From: Discussion of advanced .NET topics. [mailto:ADVANCED-
Sent: Monday, July 27, 2009 11:33 AM
Subject: [ADVANCED-DOTNET] Excess memory usage growth for a

process

Hi guys,
I need your help with an issue I am having right now. I have several

windows

services deployed. These services gather some XML files from the

internet

and then parse the content and save the information in the database.

The

database is Oracle 11g, so the access is done through ODP.NET. I had

some

issues before (and some of you with good memory may recall my previous
questions) where I noticed the memory used by these services growing

very

high. At the time, I understood that some classes in the ODP.NET

library

use

some JAVA code and they are known for not cleaning properly after

them.

As a

result I rewrote my code in such a way that every dynamic allocation

object from the ODP.NET library was encompassed in an "using"

statement,

using (OracleCommand objDbCommand = new
OracleCommand(strSqlSelectStatement,
objConnection))
All was fine for a very long time. All my services kept a constant

size

when

it came to the total memory used, a size that hovered around 55-60 MB.

This

was the case for months. At the beginning I checked the size almost on

daily basis. After a while, my confidence grew and I only checked it

once a

week or so. Every time I looked at the size of all my processes (I had

that followed pretty much the same approach on dealing with those

Oracle

objects) the size was around that 55-60 MB mark.
Now, all of a sudden, this last weekend, the size for some of the

services

shut up to about 690 MB, despite the fact the service was running with

the

exact same code as it did a month ago. This represents a potential

fatal

problem for the server that can get to a point where it can no longer
allocate memory, so I need to find a way to understand what the heck

going on. Unfortunately, I do not have any metrics implemented in my
application to log the total amount of memory used. That would have

been

very good indication on when exactly did this surge started to happen.

could have checked the Event Viewer to see if the server got hit by a
nuclear bomb or something on that precise moment. I really don't get

this. I

tools

out there that could be used to pick at the memory allocated by a

process

and see what are those objects that take up so much memory? For the

time

being, I have the processes running, but two of them are at or close

690

http://peach.ease.lsoft.com/archives

Post by Eddie Lascu
===================================
View archives and manage your subscription(s) at

http://peach.ease.lsoft.com/archives
--
Les erreurs de grammaire et de syntaxe ont été incluses pour m'assurer
de votre attention

===================================
View archives and manage your subscription(s) at
http://peach.ease.lsoft.com/archives

===================================
View archives and manage your subscription(s) at http://peach.ease.lsoft.com/archives

Greg Young

2009-07-29 16:45:57 UTC

Permalink

The wonderful thing about SOS is you dont need to run a profiler ...
when it happens take a memory dump, look whats actually happening in
the running app, is your LOH fragmented? Otherwise you are just
sticking your thumb up your butt and saying it will rain next
thursday.

Post by Eddie Lascu
I am using Reg Gate ANTS to profile the app, but it may take too much to
happen again. Before the spike I saw last weekend, the processes ran for 3-4
months with no significant change in the size of memory used.
Will look into Process.WorkingSet.
Thanks,
Eddie
-----Original Message-----
From: Discussion of advanced .NET topics.
Sent: Wednesday, July 29, 2009 12:38 PM
Subject: Re: [ADVANCED-DOTNET] Excess memory usage growth for a process
Task manager shows you working set not total memory used but yes the
process class can help you with that (Process.WorkingSet as an
example).
Have you verified the LOH being the problem by using SOS?
Cheers,
Greg

Post by Eddie Lascu
Hi guys,

http://www.simple-talk.com/dotnet/.net-framework/the-dangers-of-the-large-ob
ject-heap/

Post by Eddie Lascu
I am no longer sure what I have is a memory leak. I could see how some of

Post by Eddie Lascu
My question at this point is whether the .NET Framework has some metric

IDisposable

Post by Eddie Lascu
are being cleaned up?
Thanks,
Shawn Wildermuth
http://wildermuth.com
https://agilitrain.com
Microsoft MVP (C#), MCSD.NET, Author and Speaker
The Silverlight Tour is coming to a city near you!
-----Original Message-----
From: Discussion of advanced .NET topics.
Sent: Monday, July 27, 2009 12:28 PM
Subject: Re: [ADVANCED-DOTNET] Excess memory usage growth for a process
I don't disagree with you just note that before 'some change' working OK
no memory leaks, after change, memory leak. We both know your code did not
recompile itself to include a memory leak ...
That reminds has the framework/runtimes been updated/patched?
John Warner

a possible explanation?
I just talked with one of the DB Admin and she said she made some

changes

on
some Stored Produces, but nothing as dramatic as a patch update. She

also

said that the drivers for the network cards were updated last week, but

-----Original Message-----
From: Discussion of advanced .NET topics. [mailto:ADVANCED-
Sent: Monday, July 27, 2009 11:33 AM
Subject: [ADVANCED-DOTNET] Excess memory usage growth for a

process

Hi guys,
I need your help with an issue I am having right now. I have several

windows

services deployed. These services gather some XML files from the

internet

and then parse the content and save the information in the database.

The

database is Oracle 11g, so the access is done through ODP.NET. I had

some

issues before (and some of you with good memory may recall my previous
questions) where I noticed the memory used by these services growing

very

high. At the time, I understood that some classes in the ODP.NET

library

use

some JAVA code and they are known for not cleaning properly after

them.

As a

result I rewrote my code in such a way that every dynamic allocation

object from the ODP.NET library was encompassed in an "using"

statement,

using (OracleCommand objDbCommand = new
OracleCommand(strSqlSelectStatement,
objConnection))
All was fine for a very long time. All my services kept a constant

size

when

it came to the total memory used, a size that hovered around 55-60 MB.

This

was the case for months. At the beginning I checked the size almost on

daily basis. After a while, my confidence grew and I only checked it

once a

week or so. Every time I looked at the size of all my processes (I had

that followed pretty much the same approach on dealing with those

Oracle

objects) the size was around that 55-60 MB mark.
Now, all of a sudden, this last weekend, the size for some of the

services

shut up to about 690 MB, despite the fact the service was running with

the

exact same code as it did a month ago. This represents a potential

fatal

problem for the server that can get to a point where it can no longer
allocate memory, so I need to find a way to understand what the heck

going on. Unfortunately, I do not have any metrics implemented in my
application to log the total amount of memory used. That would have

been

very good indication on when exactly did this surge started to happen.

could have checked the Event Viewer to see if the server got hit by a
nuclear bomb or something on that precise moment. I really don't get

this. I

tools

out there that could be used to pick at the memory allocated by a

process

and see what are those objects that take up so much memory? For the

time

being, I have the processes running, but two of them are at or close

690

http://peach.ease.lsoft.com/archives

Post by Eddie Lascu
===================================
View archives and manage your subscription(s) at

http://peach.ease.lsoft.com/archives
--
Les erreurs de grammaire et de syntaxe ont été incluses pour m'assurer
de votre attention
===================================
View archives and manage your subscription(s) at
http://peach.ease.lsoft.com/archives
===================================
View archives and manage your subscription(s) at http://peach.ease.lsoft.com/archives

Eddie Lascu

2009-07-29 17:01:05 UTC

Permalink

Shoot, I didn't know SOS can do that for you. Can you share more about this
tool (a link to download it)? Is it part of the WinDbg tool that comes from
Microsoft? It's true that with ANTS I am at the mercy of the system, hoping
it will go nuts again and will have the memory size shut up crazy high.

Cheers,
Eddie

-----Original Message-----
From: Discussion of advanced .NET topics.
[mailto:ADVANCED-***@PEACH.EASE.LSOFT.COM] On Behalf Of Greg Young
Sent: Wednesday, July 29, 2009 12:46 PM
To: ADVANCED-***@PEACH.EASE.LSOFT.COM
Subject: Re: [ADVANCED-DOTNET] Excess memory usage growth for a process

The wonderful thing about SOS is you dont need to run a profiler ...
when it happens take a memory dump, look whats actually happening in
the running app, is your LOH fragmented? Otherwise you are just
sticking your thumb up your butt and saying it will rain next
thursday.

Post by Eddie Lascu
Hi guys,

http://www.simple-talk.com/dotnet/.net-framework/the-dangers-of-the-large-ob

Post by Eddie Lascu
ject-heap/

Post by Eddie Lascu
I am no longer sure what I have is a memory leak. I could see how some of

Post by Eddie Lascu
My question at this point is whether the .NET Framework has some metric

IDisposable

a possible explanation?
I just talked with one of the DB Admin and she said she made some

changes

on
some Stored Produces, but nothing as dramatic as a patch update. She

also

said that the drivers for the network cards were updated last week, but

-----Original Message-----
From: Discussion of advanced .NET topics. [mailto:ADVANCED-
Sent: Monday, July 27, 2009 11:33 AM
Subject: [ADVANCED-DOTNET] Excess memory usage growth for a

process

Hi guys,
I need your help with an issue I am having right now. I have several

windows

services deployed. These services gather some XML files from the

internet

and then parse the content and save the information in the database.

The

database is Oracle 11g, so the access is done through ODP.NET. I had

some

issues before (and some of you with good memory may recall my previous
questions) where I noticed the memory used by these services growing

very

high. At the time, I understood that some classes in the ODP.NET

library

use

some JAVA code and they are known for not cleaning properly after

them.

As a

result I rewrote my code in such a way that every dynamic allocation

object from the ODP.NET library was encompassed in an "using"

statement,

using (OracleCommand objDbCommand = new
OracleCommand(strSqlSelectStatement,
objConnection))
All was fine for a very long time. All my services kept a constant

size

when

it came to the total memory used, a size that hovered around 55-60 MB.

This

was the case for months. At the beginning I checked the size almost on

daily basis. After a while, my confidence grew and I only checked it

once a

week or so. Every time I looked at the size of all my processes (I had

that followed pretty much the same approach on dealing with those

Oracle

objects) the size was around that 55-60 MB mark.
Now, all of a sudden, this last weekend, the size for some of the

services

shut up to about 690 MB, despite the fact the service was running with

the

exact same code as it did a month ago. This represents a potential

fatal

problem for the server that can get to a point where it can no longer
allocate memory, so I need to find a way to understand what the heck

going on. Unfortunately, I do not have any metrics implemented in my
application to log the total amount of memory used. That would have

been

very good indication on when exactly did this surge started to happen.

could have checked the Event Viewer to see if the server got hit by a
nuclear bomb or something on that precise moment. I really don't get

this. I

tools

out there that could be used to pick at the memory allocated by a

process

and see what are those objects that take up so much memory? For the

time

being, I have the processes running, but two of them are at or close

690

http://peach.ease.lsoft.com/archives

Post by Eddie Lascu
===================================
View archives and manage your subscription(s) at

http://peach.ease.lsoft.com/archives
--
Les erreurs de grammaire et de syntaxe ont été incluses pour m'assurer
de votre attention
===================================
View archives and manage your subscription(s) at
http://peach.ease.lsoft.com/archives
===================================
View archives and manage your subscription(s) at

Greg Young

2009-07-29 17:06:11 UTC

Permalink

Yes with ants you have to wait for the problem to happen.

if you google SOS you can find some tutorials on it (it is a separate
download from windbg ... it can also be used in visual studio!). John
Robbins also has a great book covering the subject that I highly
recommend.

Cheers,

greg

Post by Eddie Lascu
Shoot, I didn't know SOS can do that for you. Can you share more about this
tool (a link to download it)? Is it part of the WinDbg tool that comes from
Microsoft? It's true that with ANTS I am at the mercy of the system, hoping
it will go nuts again and will have the memory size shut up crazy high.
Cheers,
Eddie
-----Original Message-----
From: Discussion of advanced .NET topics.
Sent: Wednesday, July 29, 2009 12:46 PM
Subject: Re: [ADVANCED-DOTNET] Excess memory usage growth for a process
The wonderful thing about SOS is you dont need to run a profiler ...
when it happens take a memory dump, look whats actually happening in
the running app, is your LOH fragmented? Otherwise you are just
sticking your thumb up your butt and saying it will rain next
thursday.

Post by Eddie Lascu
I am using Reg Gate ANTS to profile the app, but it may take too much to
happen again. Before the spike I saw last weekend, the processes ran for

3-4

Post by Eddie Lascu
months with no significant change in the size of memory used.
Will look into Process.WorkingSet.
Thanks,
Eddie
-----Original Message-----
From: Discussion of advanced .NET topics.
Sent: Wednesday, July 29, 2009 12:38 PM
Subject: Re: [ADVANCED-DOTNET] Excess memory usage growth for a process
Task manager shows you working set not total memory used but yes the
process class can help you with that (Process.WorkingSet as an
example).
Have you verified the LOH being the problem by using SOS?
Cheers,
Greg

Post by Eddie Lascu
Hi guys,

http://www.simple-talk.com/dotnet/.net-framework/the-dangers-of-the-large-ob

Post by Eddie Lascu
ject-heap/

Post by Eddie Lascu
I am no longer sure what I have is a memory leak. I could see how some of

this

Post by Eddie Lascu
excess memory usage that I experience every once in a blue moon. What I
thought was to monitor the memory used and terminate the application in

case

Post by Eddie Lascu
it exceeds a set threshold. The Windows Service Manager will then restart
the service and all the memory will be reclaimed by the OS.

Post by Eddie Lascu
My question at this point is whether the .NET Framework has some metric

classes that can help monitor the total amount of memory allocated within

Post by Eddie Lascu
process. Surely System.Diagnostics must have something like that. Is

anyone

Post by Eddie Lascu
aware of a way to find out what memory is used by a process, just like the
Task Manager does?

Post by Eddie Lascu
Cheers,
Eddie
-----Original Message-----
From: Discussion of advanced .NET topics.

Wildermuth