capacity management

[TdC]

if you had a chance to have something monitored on your servers that wasn't cpu memory or disk usage, what would it be?

 

[inactionman]

Assume you are including swap file usage? you prob want things like Disk I/O, Network utilisation, and Temperature.

 

[TdC]

disk IO will be in the realm of performance management I guess. I *am* looking at total_disk_space, total_filesystem_space, used_filesystem_space, free_filesystem_space

I'm going to do network IO because I can only get total io through switches and not the separate nodes, and swap usage because while I am doing memory I'm not including virtual as yet.

 

[inactionman]

You can't get network card utilisation from each server? That's odd! You should either be able to get it from the server or the switch (definitely last time I was working on Cisco switches you could get both the backplane utilisation, and for each port). What type switches are you using?

 

[yaruar]

disk IO will be in the realm of performance management I guess. I *am* looking at total_disk_space, total_filesystem_space, used_filesystem_space, free_filesystem_space

I'm going to do network IO because I can only get total io through switches and not the separate nodes, and swap usage because while I am doing memory I'm not including virtual as yet.

Definately disk free space.

Also network utilisation. Would depend on what the server was doing. For example with the exchage servers i manager we monitor message queue size, and SMTP calls load across the servers. For database servers i'd definately monitor sql queries, writes, ets.

Usual performance indicators like cache writes/reads, dirty or otherwise, memory usage, etc.

 

[TdC]

You can't get network card utilisation from each server? That's odd! You should either be able to get it from the server or the switch (definitely last time I was working on Cisco switches you could get both the backplane utilisation, and for each port). What type switches are you using?

Ciscos, but the dept that admins them doesn't do per-port stats, just the backplanes.

Also, I most assuredly can get it off my suns, it's just that we don't do it as a dept and I amd the capacity bloke and want it

 

[TdC]

Definately disk free space.

Also network utilisation. Would depend on what the server was doing. For example with the exchage servers i manager we monitor message queue size, and SMTP calls load across the servers. For database servers i'd definately monitor sql queries, writes, ets.

Usual performance indicators like cache writes/reads, dirty or otherwise, memory usage, etc.

free space yes, most of the other stuff is in the realm of performance monitoring. also, I can't be too specfic with what I monitor, because I admin 1700odd suns and I really cba to be that specific for each one

 

[yaruar]

free space yes, most of the other stuff is in the realm of performance monitoring. also, I can't be too specfic with what I monitor, because I admin 1700odd suns and I really cba to be that specific for each one

With that many i'd just make sure they respond to ping and whatever service they are running is actually up

 

[inactionman]

I'm always of the opinion to capture as much as you can, then filter the reports for what you need (as if data is never captured, it's lost). So I'd capture all the information yaruar suggests, and just filter it out of your reports (maybe giving the performance guys access to it so they can run their own reports), but maybe that's the IDS guy in me speaking (the more information you have the better to investigate/correlate things).

 

[inactionman]

Ciscos, but the dept that admins them doesn't do per-port stats, just the backplanes.

And that's stupid, it generates less traffic for their management system to push the data to you, than for you to pull it from the servers, and I can pretty much guarantee they are capturing it anyway!

 

[mank!]

A live scorecard of England v Sri Lanka.

 

[Wij]

NIC.

Plus some kind of connectivity test to whatever uses it.

 

[Chilly]

http://munin.primefarm.co.uk/

Thats my ghetto monitoring - might find it helpful.

 

[TdC]

cheers, though tbh a lot of that is in the realm of performance monitoring and not capacity particularly. Also, some things are in the realm of proactively monitoring for incidents, ie watching your filesystems for excessive growth. for example, our first line gets triggered whenever any filesystem grows over its highwater mark as is, so it's not particualrly a capacity issue. (that would be more like used-filesystems nearing configured-filessytems which in turn would have to near available-disk-size

this is what I currently check through the proggy I bodged:

1175700224 SUN suntest CPU 32 26 1 42 2
1175700224 SUN suntest CPU 0 38 25 0 38
1175700224 SUN suntest CPU 1 38 25 0 38
1175700224 SUN suntest MEM 2147483648 1017528320 1129955328 374792192
1175700224 SUN suntest DISKS 946883478528 372559426 146482382 2147737600

this translates as

timestamp OS hostname object value1 value2 value3 enz

timestamp SUN suntest CPU usr sys wio idl numcpu (totals)
timestamp SUN suntest CPU num usr sys wio idl
timestamp SUN suntest CPU num usr sys wio idl
timestamp SUN suntest MEM actual used free swap_in_ram
timestamp SUN suntest DISKS actual conf_fs used_fs swap_on_disk

ofc if you have 72 cpus then there will be 73 lines of cpu etc.

I am going to add NICs a bit like the cpu thing; totals and then bytes in and out for each nic. I am considering a line on swapspace by itself, with just total configured and used_space, and the swap_in_ram added to reveal total_used_virt, but atm the swap to ram is in the memory line and the configured swap on disk is in the disk line.

 

[evzy]

Hmmm no idea why I clicked on this thread... It just read blah blah blah blah Sri Lanka blah blah blah to me.

 

[TdC]

I know. sometimes it melts my brains also