You want to fetch web pages, but the server is using cookies to track you. For example, some sites use a cookie to remember that you've authenticated. If you don't send the right cookie, you'll never get past the login screen.
Let LWP::UserAgent handle cookies for you. You can enable cookies for just this program run with:
$ua->cookie_jar({ });
Or instead store cookies in a file between invocations with:
$ua->cookie_jar({ file => "$ENV{HOME}/.cookies" });
The default behavior of LWP::UserAgent is never to send a Cookie: header, even when the server offers cookies in a response. To keep track of cookies LWP::UserAgent receives and send them when appropriate, provide the user agent object with a special "cookie jar" object to hold the cookies: an HTTP::Cookies object.
Pass the cookie_jar method either an HTTP::Cookies object to use that object as the cookie jar, or else a hash reference whose contents go into a new HTTP::Cookies object.
Without parameters, an HTTP::Cookies object keeps cookies in memory, so they're no longer available once your program exits. The file parameter in the cookie_jar method call specifies a filename to use for initializing the cookie jar and for saving updated or new cookies. This is how you give cookies a shelf life beyond a single run of your program.
To disable cookies, call cookie_jar with no parameters:
$ua->cookie_jar( );
The documentation for the CPAN modules LWP::UserAgent and HTTP::Cookie
Copyright © 2003 O'Reilly & Associates. All rights reserved.