openssl segmentation fault

Bugs item #8903, was opened at 2007-02-27 01:08
You can respond by visiting: 
http://rubyforge.org/tracke
r/?func=detail&atid=1698&aid=8903&group_id=426

Category: None
Group: None
Status: Open
Resolution: None
Priority: 3
Submitted By: Mark Gallop (markg)
Assigned to: Nobody (None)
Summary: openssl segmentation fault

Initial Comment:
I am not sure if this should be reported but I am getting a
segmentation fault with the final line of the following
code:

require 'openssl'
require 'digest/sha2'
c = OpenSSL::Cipher::Cipher.new("aes-256-cbc")
c.encrypt
c.key = key = Digest::SHA2.digest("yourpass")
c.iv = iv = c.random_iv
e = c.update("crypt this")
OpenSSL:ebug =
true
e << c.final

(irb):9: [BUG] Segmentation fault

I realise that it is a typo to have
Digest::SHA2.digest("yourpass") and not
Digest::SHA256.digest("yourpass") but I figure it
still shouldn't segfault.

This happens with these systems/versions

Mac OS X - ruby 1.8.5 (2006-12-25) [powerpc-darwin8.8.0],
OpenSSL 0.9.8d
Gentoo - ruby 1.8.5 (2006-12-04) [x86_64-linux], OpenSSL
0.9.8d

but does not segfault with:

Ubuntu - ruby 1.8.4 (2005-12-24) [i486-linux], OpenSSL
0.9.8b

Cheers,
Mark

------------------------------------------------------------
----------

You can respond by visiting: 
http://rubyforge.org/tracke
r/?func=detail&atid=1698&aid=8903&group_id=426

On Tue, Feb 27, 2007 at 10:08:44AM +0900, noreplyrubyforge.org wrote:
> Initial Comment:
> I am not sure if this should be reported but I am
getting a segmentation fault with the final line of the
following code:
> 
> require 'openssl'
> require 'digest/sha2'
> c =
OpenSSL::Cipher::Cipher.new("aes-256-cbc")
> c.encrypt
> c.key = key =
Digest::SHA2.digest("yourpass")
> c.iv = iv = c.random_iv
> e = c.update("crypt this")
> OpenSSL:ebug =
true
> e << c.final
> 
> (irb):9: [BUG] Segmentation fault
> 
> I realise that it is a typo to have
Digest::SHA2.digest("yourpass") and not
Digest::SHA256.digest("yourpass") but I figure it
still shouldn't segfault.

I can't reproduce on my system, Ubuntu, with openssl 0.9.8b
04 May 2006,
and ruby 1.8.6 (2007-02-28 patchlevel 5000) [i686-linux].

Could you provide the stack backtrace from gdb? Do

 gdb <your ruby> core
 > bt


SHA2 is same as SHA256, they result in the same 32 byte key,
so I don't
see how that is what is causing this. Are you saying if you
use
Digest::SHA256 you *don't* get a segfault? What about if you
use "x"*32
as a key?


Maybe you know, but running a straight digest on a password
is not a
recomended way of generating a key from a pass phrase, it is
quite
easily subject to attack. You might consider using
Cipher#pkcs5_keyivgen().

Sam