fetch an HTML page from a secure site that requires authorization

On Wed, March 5, 2008 08:46, David Lowrey wrote:
> Andrej,<br>
> <br>
> Format your URL like this:<br>
> <a class="moz-txt-link-freetext"
> href="http://userid:passwordhost/path">http://userid:passwordhost/path</a><br>
> <br>
> The only problem with this approach is that your userid
&amp; password
> are sent out in clear text...<br>
> <br>
> Dave<br>
> <br>
> Andrej Sokurov wrote:
> <blockquote
> 
cite="mid:73258fd80803050759s5fbc3f05l426728f371420db1<
img src="/img/at.gif" align="middle" border="0"
alt="">mail.gmail.com"
>  type="cite">Hello everyone:<br>
>   <br>
> I need to write a program that would fetch an HTML page
from a secure
> site that requires authorization.<br>
> When a human it usually done in following
sequence:<br>
> 1. Get login
screen&nbsp;&nbsp;&nbsp;&nbsp;
(https)<br>
> 2. Supply login/password<br>
> 3. Get needed HTML page (https)<br>
> 4. Logout<br>
>   <br>
> Doing similar thing without required authorization is
very simple and
> straightforward procedure:<br>
>   <br>
> &nbsp; my($request) = new HTTP::Request GET
=&gt; $url;<br>
> &nbsp; my($ua) = new LWP::UserAgent;<br>
> &nbsp; my($response) =
$ua-&gt;request($request);<br>
> &nbsp;
my($html_page_received)=$response-&gt;content;<br>

>   <br>
> However, getting through secure site that requires
authorization is not
> clear for me.<br>
> Could somebody point me in a right direction? It would
be the best, if
> somebody gives me an example of such program<br>
>   <br>
> Thank you.<br>
>   <pre wrap="">
> <hr size="4" width="90%">
Actually I think that you need to make two exchanges, the
first deals with
the login, the second exchanges the information.   If you
are going to an
https page I think LWP will use standard SSL protocol so the
username and
password are NOT sent in the clear.

Bill

_______________________________________________
Perl-Win32-Web mailing list
Perl-Win32-Weblistserv.ActiveState.com
To unsubscribe: http:/
/listserv.ActiveState.com/mailman/mysubs

On Fri, March 7, 2008 11:16, Andrej Sokurov wrote:
> Thank you to everyone for helping me out.
>
> I might be mistaken, but I have an impression that
there are 2 kind of
> authorizations.
>
> 1) It looks like in example below:
>    htt
p://www.buddhism-dict.net/cgi-bin/xpr-dealt.pl
>    There is an example for perl program here:
>
> http://www.rosettacode
.org/rosettacode/w/index.php?title=HTTPS_request_with_authen
tication
>
> 2) When there is a login screen where a user needs to
supply
> login/password.
> It's similar to PayPal login screen.
>
> I'm looking solutions for the second type of
authorization.
>
> I wrote a program, based on examples received from this
post. It doesn't
> work. I tried to login into a regular,
> not https site ... but failed.
>
> I don't have publicly available site and guests login
to share with the
> community ... unfortunately.
>
> The task seems straightforward:
> 1. Posting login/password information on proper fields
of the form.
> 2. Requesting needed page.
>
> It looks like POST works OK (Block 1).
> In block #2 I'm getting back login screen and saving
into file for further
> review. It looks like login/password information
> is correct when I looked source code for this HTML.
>
> Block #3 returns me HTML page but not that I expecting.
It returns the
> page
> that normally comes when a user tried to get
> this page directly, without authorization or with
invalid session object.
>
> Does anybody have suggestion what I'm doing wrong and
how to fix it?
>
> Thank you
>
Check to see if its setting a cookie. You may have to send
that back with
your next request. Just a thought.
>
>
#----------------------------------------------------------
code start
> use LWP::UserAgent;
> use HTTP::Cookies;
> use HTTP::Request::Common qw(GET POST);
> use strict;
>
>
> my $url='http://a_site/login.jsp';

>
> my $ua = LWP::UserAgent->new();
> $ua->agent('Mozilla/5.0');
>
> my $cookie_jar = HTTP::Cookies->new();
>
> my fld_val  =();
> my fld_name =();
>
> $fld_name[0]='user_name';   $fld_val[0]= 'Tom';
> $fld_name[1]='password';    $fld_val[1]=
'pass_for_tom';
>
> my $data = [ $fld_name[0] => $fld_val[0],
>              $fld_name[1] => $fld_val[1]
>        ];
>
>
> #----------------- Block 1 -------------------
> my $response = $ua->request(POST($url, Content =>
$data ) );
>
> if ($response->is_error )
>   {
>     die "Response ERROR";
>   };
>
> $cookie_jar->extract_cookies($response);
> $cookie_jar->save('cookie_01.txt');
> #-----
>
>
> #----------------- Block 2 -------------------
> # Still login screen
> my $ct = $response->content;
>
> my $file_name='U:PerlDataMisclog_01.html';
> open(FILE_LOG, ">$file_name") ||
die"Can not open file $file_name to
> write";
> print FILE_LOG "$ct";
> close(FILE_LOG);
> #---
>
>
> #----------------- Block 3 -------------------
> # Below is a page I need to get
> $url = 'http://a_site/main.jsp';
> my $request = new HTTP::Request GET => $url;
> $response = $ua->request($request);
>
> my $html_page_received='';
> $html_page_received = $response->content;
>
>
> $file_name='U:PerlDataMisctarget_01.html';
> open(FILE_TRG, ">$file_name") ||
die"Can not open file $file_name to
> write";
> print FILE_TRG "$html_page_received";
> close(FILE_TRG);
>
#----------------------------------------------------------
code end
>
>
> On Wed, Mar 5, 2008 at 10:59 AM, Andrej Sokurov
<archive1600gmail.com>
> wrote:
>
>> Hello everyone:
>>
>> I need to write a program that would fetch an HTML
page from a secure
>> site
>> that requires authorization.
>> When a human it usually done in following
sequence:
>> 1. Get login screen     (https)
>> 2. Supply login/password
>> 3. Get needed HTML page (https)
>> 4. Logout
>>
>> Doing similar thing without required authorization
is very simple and
>> straightforward procedure:
>>
>>   my($request) = new HTTP::Request GET => $url;
>>   my($ua) = new LWP::UserAgent;
>>   my($response) = $ua->request($request);
>>   my($html_page_received)=$response->content;
>>
>> However, getting through secure site that requires
authorization is not
>> clear for me.
>> Could somebody point me in a right direction? It
would be the best, if
>> somebody gives me an example of such program
>>
>> Thank you.
>>
> _______________________________________________
> Perl-Win32-Web mailing list
> Perl-Win32-Weblistserv.ActiveState.com
> To unsubscribe: http:/
/listserv.ActiveState.com/mailman/mysubs
>


_______________________________________________
Perl-Win32-Web mailing list
Perl-Win32-Weblistserv.ActiveState.com
To unsubscribe: http:/
/listserv.ActiveState.com/mailman/mysubs