These forums have been archived and are now read-only.

The new forums are live and can be found at https://forums.eveonline.com/

EVE Technology Lab

 
  • Topic is locked indefinitely.
 

Problems parsing Chatlogs with Perl

Author
Chuck Turing
Solarmark
Stellarium Alliance
#1 - 2013-04-29 11:02:41 UTC
Hi,

I am trying to parse Chatlogs files with Perl, but I am having difficulties to decode properly the contents.

I know I could use the external tool iconv to pre-convert them, but I want to do something that runs on Windows & Unix, so all pure Perl code.

I have tried opening the file with the following methods, but all failed and I keep getting BOMs in the text like: "[ 2012.08.26 01:05:35 ]"

Tests:

$path='corp.txt'; # file to parse

use File::Slurp::Unicode;
my @lines = read_file( $path,encoding => 'UTF-16LE') ;

or

open(my $in_file,"<:raw:perlio:encoding(UTF-16le):crlf:utf8", $path) ;
@lines=<$in_file>;
close ($in_file);

And also tried the function from_to() of Encode module.

Any hint?

Thanks in advance.
Chuck Turing
Solarmark
Stellarium Alliance
#2 - 2013-04-29 13:12:28 UTC
Update:

Finally I opted for:

open(my $in_file,"<:raw:perlio:encoding(UTF-16le):crlf:utf8", $path) or die("Can't open \"$path\": $!\n");
my @lines=<$in_file>;
close ($in_file);

And chomping each line.

My regexp ignores the starting BOM.

If there's any other suggestion, it would be appreciated.