ASA 5510 initial setup. Can't reach the outside from inside.

I think you have to configure a rule that allows ICMP through the interface...

When using the ASDM and pinging from the outside interface I can ping and get a response from the internet, but when trying to ping the internet from the inside interface I'm not getting any response.

The question didn't state that this was happening from an inside client as well.

Anyhow, skepticals is correct. Although the Pix/ASA is a stateful firewall, this only appears to be for TCP/UDP traffic. You will find that although you allow ICMP traffic out of one interface, the return traffic will not automatically return unless you add the echo-reply to the access-list. If you create an access-list on the external to allow echo-replies then it should start to work.

I have this in my config:

[code:1]access-list Static extended permit icmp any any echo-reply
access-list Static extended permit icmp any any unreachable[/code:1]

Try this command,

access-list icmp_ping extended permit icmp any any echo-reply
access-group icmp_ping in interface outside
access-group icmp_ping in interface inside

Let me know if this works.