我在Perl中有一些记忆问题.当我填满一个大的哈希时,我无法将内存释放回操作系统.当我使用标量执行相同操作并使用undef时,会将内存恢复到操作系统.
这是我写的一个测试程序.
#!/usr/bin/perl
###### Memory test
######
## Use Commands
use Number::Bytes::Human qw(format_bytes);
use Data::Dumper;
use Devel::Size qw(size total_size);
## Create Varable
my $share_var;
my %share_hash;
my $type_hash = 1;
my $type_scalar = 1;
## Start Main Loop
while (true) {
&Memory_Check();
print "Hit Enter (add to memory): "; <>;
&Up_Mem(100_000);
&Memory_Check();
print "Hit Enter (Set Varable to nothing): "; <>;
$share_var = "";
$share_hash = ();
&Memory_Check();
print "Hit Enter (clean data): "; <>;
&Clean_Data();
&Memory_Check();
print "Hit Enter (start over): "; <>;
}
exit;
#### Up Memory
sub Up_Mem {
my $total_loops = shift;
my $n = 1;
print "Adding data to shared varable $total_loops times\n";
until ($n > $total_loops) {
if ($type_hash) {
$share_hash{$n} = 'X' x 1111;
}
if ($type_scalar) {
$share_var .= 'X' x 1111;
}
$n += 1;
}
print "Done Adding Data\n";
}
#### Clean up Data
sub Clean_Data {
print "Clean Up Data\n";
if ($type_hash) {
## Method to fix hash (Trying Everything i can think of!
my $n = 1;
my $total_loops = 100_000;
until ($n > $total_loops) {
undef $share_hash{$n};
$n += 1;
}
%share_hash = ();
$share_hash = ();
undef $share_hash;
undef %share_hash;
}
if ($type_scalar) {
undef $share_var;
}
}
#### Check Memory Usage
sub Memory_Check {
## Get current memory from shell
my @mem = `ps aux | grep \"$$\"`;
my($results) = grep !/grep/,@mem;
## Parse Data from Shell
chomp $results;
$results =~ s/^\w*\s*\d*\s*\d*\.\d*\s*\d*\.\d*\s*//g; $results =~ s/pts.*//g;
my ($vsz,$RSS) = split(/\s+/,$results);
## Format Numbers to Human Readable
my $h = Number::Bytes::Human->new();
my $virt = $h->format($vsz);
my $h = Number::Bytes::Human->new();
my $res = $h->format($RSS);
print "Current Memory Usage: Virt: $virt RES: $res\n";
if ($type_hash) {
my $total_size = total_size(\%share_hash);
my @arr_c = keys %share_hash;
print "Length of Hash: " . ($#arr_c + 1) . " Hash Mem Total Size: $total_size\n";
}
if ($type_scalar) {
my $total_size = total_size($share_var);
print "Length of Scalar: " . length($share_var) . " Scalar Mem Total Size: $total_size\n";
}
}
OUTPUT：
./Memory_Undef_Simple.cgi Current Memory Usage: Virt: 6.9K RES: 2.7K Length of Hash: 0 Hash Mem Total Size: 92 Length of Scalar: 0 Scalar Mem Total Size: 12 Hit Enter (add to memory): Adding data to shared varable 100000 times Done Adding Data Current Memory Usage: Virt: 228K RES: 224K Length of Hash: 100000 Hash Mem Total Size: 116813243 Length of Scalar: 111100000 Scalar Mem Total Size: 111100028 Hit Enter (Set Varable to nothing): Current Memory Usage: Virt: 228K RES: 224K Length of Hash: 100000 Hash Mem Total Size: 116813243 Length of Scalar: 0 Scalar Mem Total Size: 111100028 Hit Enter (clean data): Clean Up Data Current Memory Usage: Virt: 139K RES: 135K Length of Hash: 0 Hash Mem Total Size: 92 Length of Scalar: 0 Scalar Mem Total Size: 24 Hit Enter (start over):
所以,你可以看到内存下降,但它只会降低标量的大小.任何想法如何释放哈希的内存？
另外Devel::Size显示,即使程序仍在使用139K,散列仅占用92个字节.
解决方法
一般来说,是的,这就是UNIX的内存管理工作原理.如果您正在使用最新的glibc,并且正在使用该malloc,您可以将free’d内存返回到操作系统.我不知道Perl是否这样做.
如果要使用大型数据集,请勿将整个内容加载到内存中,使用像BerkeleyDB这样的东西：
https://metacpan.org/pod/BerkeleyDB
use strict ;
use BerkeleyDB ;
my $filename = "fruit" ;
unlink $filename ;
tie my %h,"BerkeleyDB::Hash",-Filename => $filename,-Flags => DB_CREATE
or die "Cannot open file $filename: $! $BerkeleyDB::Error\n" ;
# Add a few key/value pairs to the file
$h{apple} = "red" ;
$h{orange} = "orange" ;
$h{banana} = "yellow" ;
$h{tomato} = "red" ;
# Check for existence of a key
print "Banana Exists\n\n" if $h{banana} ;
# Delete a key/value pair.
delete $h{apple} ;
# print the contents of the file
while (my ($k,$v) = each %h)
{ print "$k -> $v\n" }
untie %h ;
(好的,不是逐字的,他们使用vars是…遗产…)
您可以通过这种方式将哈字节数据存储在哈希中,您只能使用一小段内存. (基本上,无论BDB的寻呼机决定保持记忆,这是可控的.)