Hoptoad not receiving errors from production
Account subdomain: onecause
I'm trying to set up Hoptoad again, but the same issue seems to be present as when I tried it last year. I thought perhaps a recent upgrade to Rails 2.3.2 would fix things, but that's not the case.
rake hoptoad:test works fine in development and staging, but not on our production machines. The output is the same in any environment, though:
$ rake hoptoad:test
(in /home/onecause/onecause)
Setting up the Controller.
Processing request.
Raising 'HoptoadTestingException' to simulate application failure.
I ran the code in the rake task in the console in production, however, and this was the response after the "HoptoadVerificationController.new.process(request, response)" line:
HoptoadVerificationController.new.process(request, response)
Raising 'HoptoadTestingException' to simulate application failure.
=> #<ActionController::TestResponse:0x2aaab31dab40 @redirected_to=nil, @body="<!DOCTYPE html PUBLIC \"-//W3C//DTD XHTML 1.0 Transitional//EN\" \"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd\">\r\n<html xmlns=\"http://www.w3.org/1999/xhtml\">\r\n <head>\r\n <title>Error Page: We're sorry, but something went wrong</title><!-- page title followed by global title -->\r\n <meta content=\"text/html; charset=utf-8\" http-equiv=\"Content-Type\" />\r\n <link href=\"/stylesheets/initial.css\" media=\"screen\" rel=\"stylesheet\" type=\"text/css\" />\r\n<link href=\"/stylesheets/basic_typography.css\" media=\"screen\" rel=\"stylesheet\" type=\"text/css\" />\r\n<link href=\"/stylesheets/menus.css\" media=\"screen\" rel=\"stylesheet\" type=\"text/css\" />\r\n<link href=\"/stylesheets/modules.css\" media=\"screen\" rel=\"stylesheet\" type=\"text/css\" />\r\n<link href=\"/stylesheets/layout.css\" media=\"screen\" rel=\"stylesheet\" type=\"text/css\" />\r\n<link href=\"/stylesheets/common.css\" media=\"screen\" rel=\"stylesheet\" type=\"text/css\" />\r\n <!--[if IE 6]>\r\n<link href=\"/stylesheets/ie6.css\" rel=\"stylesheet\" type=\"text/css\" />\r\n<![endif]--> \r\n </head>\r\n\r\n <body>\r\n <div id=\"wrap\">\r\n <div id=\"wrapOuter\">\r\n <div id=\"wrapInner\">\r\n <div id=\"mainBody\"><!-- contains header, main content and footer -->\r\n <div id=\"header\">\r\n <div id=\"topBar\">\r\n <div id=\"memberUtilityLinks\">\r\n <ul class=\"utilityMenu\">\r\n <li><a href=\"/signup\" class=\"menuFirst\">Sign up</a></li>\r\n <li><a href=\"/signin\">Log in</a></li>\r\n <li> <a href=\"http://help.onecause.com/\">Help</a></li>\r\n </ul>\r\n </div>\r\n <div id=\"logo\">\r\n <a href=\"/\" title=\"OneCause Home\"><img alt=\"\" border=\"0\" height=\"53\" src=\"http://www.onecause.com/images/onecause_logo.gif\" width=\"240\" /></a>\r\n </div>\r\n <div id=\"memberCause_testimonial\">\r\n\r\n <!-- if the user is not logged in or cookied, show the testimonial -->\r\n <div id=\"testimonial\">\r\n <span class=\"testimonialBig\">\"It's so easy!</span> You can earn money for your cause by doing <br /><span class=\"testimonialIndent\">what you already do -- shopping with your favorite merchants.</span><span class=\"testimonialBig\">\"</span>\r\n </div>\r\n\r\n </div><!-- end memberCause_testimonial -->\r\n <br class=\"clear\"/> \r\n </div><!-- end of topBar -->\r\n \r\n <div id=\"menuBarOuter\">\r\n <div id=\"menuBar\">\r\n <span class=\"x1\"><span class=\"x1a\"></span></span>\r\n <div id=\"mainMenu\">\r\n <ul id=\"\">\r\n \r\n <li id=\"home\" class='tab'><a href=\"/\">Home</a></li>\r\n \r\n <li id=\"shop\" ><a href=\"/shop\" title=\"\">Shop</a></li>\r\n <li id=\"causes\" ><a href=\"/causes\" title=\"\">Causes</a></li>\r\n <li id=\"schools\" ><a href=\"/schools\">Schools</a></li>\r\n\r\n </ul>\r\n </div><!-- end mainMenu -->\r\n \r\n <div id=\"subMenu\">\r\n\r\n </div> <!-- end subMenu -->\r\n \r\n <span class=\"x2\"><span class=\"x2a\"></span></span> \r\n </div><!-- end of menuBar -->\r\n </div><!-- end of menuBarOuter -->\r\n <br class=\"clear\"/>\r\n</div><!-- end of header -->\r\n\r\n <div id=\"mainPage\"><!-- contains main content, left and right columns -->\r\n <div id=\"systemMessages\"></div>\r\n \r\n <div id=\"content\"><!-- contains main content and right -->\r\n <div id=\"contentFull\">\r\n <div class=\"dialog textCenter\">\r\n <h1>We're sorry, but something went wrong.</h1>\r\n <p>We have been notified of this issue and will address it shortly.</p>\r\n </div>\r\n \r\n </div><!-- end of contentFull/Center/Left -->\r\n \r\n <!--<div id=\"rightColumn\"></div><br class=\"clear\"/> -->\r\n <!-- end of right column -->\r\n </div><!-- end of content -->\r\n\r\n\r\n </div><!-- end of mainpage -->\r\n <br class=\"clear\" />\r\n <div id=\"footer\">\r\n <div id=\"footerMenu\">\r\n\r\n <p class=\"textCenter\"><a href=\"http://www.onecause.com/?lid=footer_menu\" title=\"OneCause Home\">Home</a> | <a href=\"http://www.onecause.com/shop?lid=footer_menu\" title=\"Shop\">Shop</a> | <a href=\"http://www.onecause.com/causes?lid=footer_menu\" title=\"Causes\">Causes</a> | <a href=\"http://www.onecause.com/schools?lid=footer_menu\" title=\"Schools\">Schools</a> | \r\n <a href=\"http://help.onecause.com?lid=footer_menu\" title=\"Help\">Help</a> | <a href=\"http://www.onecause.com/sitemap.html?lid=footer_menu\" title=\"Sitemap\">Sitemap</a>\r\n\r\n </p>\r\n \r\n <p class=\"textCenter\">\r\n <a href=\"http://help.onecause.com/index.php?option=com_content&view=category&layout=blog&id=1&Itemid=2&lid=footer_menu\" title=\"About Us\">About Us</a> | \r\n <a href=\"http://help.onecause.com/index.php?option=com_content&view=category&layout=blog&id=2&Itemid=3&lid=footer_menu\" title=\"Programs\">Programs</a> | \r\n <a href=\"http://www.onecause.com/advertise?lid=footer_menu\" title=\"Advertise\">Advertise</a> | \r\n <a href=\"http://help.onecause.com/index.php?option=com_content&view=article&id=9&Itemid=5&lid=footer_menu\" title=\"Terms of Service\">Terms of Service</a> | <a href=\"http://help.onecause.com/index.php?option=com_content&view=article&id=50&Itemid=38&lid=footer_menu\" title=\"Terms of Use\">Terms of Use</a> | \r\n <a href=\"http://help.onecause.com/index.php?option=com_content&view=article&id=8&Itemid=6&lid=footer_menu\" title=\"Privacy Policy\">Privacy Policy</a> | \r\n <a href=\"http://help.onecause.com/index.php?option=com_content&view=article&id=10&Itemid=7&lid=footer_menu\" title=\"Contact Us\">Contact Us</a>\r\n\r\n </p>\r\n </div><br/>\r\n <div id=\"copyright\">© 1999-2008 CauseLoyalty, LLC. All Rights Reserved.</div>\r\n</div><!-- end of footer --> \r\n\r\n </div><!-- end of mainbody --> \r\n </div><!-- end of wrapInner -->\r\n </div><!-- end of wrapOuter -->\r\n </div><!-- end of wrap- -->\r\n <script type=\"text/javascript\">\r\n var gaJsHost = ((\"https:\" == document.location.protocol) ? \"https://ssl.\" : \"http://www.\");\r\n document.write(unescape(\"%3Cscript src='\" + gaJsHost + \"google-analytics.com/ga.js' type='text/javascript'%3E%3C/script%3E\"));\r\n </script>\r\n <script type=\"text/javascript\">\r\n var pageTracker = _gat._getTracker(\"UA-5307838-1\");\r\n pageTracker._trackPageview();\r\n </script>\r\n\r\n </body>\r\n</html>\r\n", @request=#<ActionController::TestRequest:0x2aaab3203810 @request_uri="/", @env={"SERVER_NAME"=>"example.org", "HTTP_USER_AGENT"=>"Rails Testing", "rack.run_once"=>false, "rack.input"=>#<StringIO:0x2aaab31dfca8>, "PATH_INFO"=>"/", "rack.url_scheme"=>"http", "CONTENT_LENGTH"=>"0", "action_controller.request.request_parameters"=>{}, "rack.errors"=>#<StringIO:0x2aaab31dfd48>, "SCRIPT_NAME"=>"", "REMOTE_ADDR"=>"0.0.0.0", "rack.multithread"=>true, "rack.version"=>[0, 1], "rack.routing_args"=>{}, "rack.multiprocess"=>true, "SERVER_PORT"=>80, "QUERY_STRING"=>"", "REQUEST_METHOD"=>"GET"}, @cookies={}, @host="test.host", @parameters={}, @session={}, @request_method=:get, @content_type=nil, @query_parameters={}>, @session={}, @template=#<ActionView::Base:0x2aaab3389658 @controller=#<HoptoadVerificationController:0x2aaab3389950 @db_rt_after_render=0, @url=#<ActionController::UrlRewriter:0x2aaab3387a88 @request=#<ActionController::TestRequest:0x2aaab3203810 @request_uri="/", @env={"SERVER_NAME"=>"example.org", "HTTP_USER_AGENT"=>"Rails Testing", "rack.run_once"=>false, "rack.input"=>#<StringIO:0x2aaab31dfca8>, "PATH_INFO"=>"/", "rack.url_scheme"=>"http", "CONTENT_LENGTH"=>"0", "action_controller.request.request_parameters"=>{}, "rack.errors"=>#<StringIO:0x2aaab31dfd48>, "SCRIPT_NAME"=>"", "REMOTE_ADDR"=>"0.0.0.0", "rack.multithread"=>true, "rack.version"=>[0, 1], "rack.routing_args"=>{}, "rack.multiprocess"=>true, "SERVER_PORT"=>80, "QUERY_STRING"=>"", "REQUEST_METHOD"=>"GET"}, @cookies={}, @host="test.host", @parameters={}, @session={}, @request_method=:get, @content_type=nil, @query_parameters={}>, @parameters={}>, @request_origin="0.0.0.0 at 2009-08-11 14:21:38", @_headers={"Content-Type"=>"text/html; charset=utf-8", "Cache-Control"=>"no-cache", "Set-Cookie"=>[], "Content-Length"=>"6367"}, @db_rt_before_render=0, @performed_render=true, @_response=#<ActionController::TestResponse:0x2aaab31dab40 ...>, @_session={}, @action_name="index", @template=#<ActionView::Base:0x2aaab3389658 ...>, @_params={}, @performed_redirect=false, @view_runtime=0.982046127319336, @before_filter_chain_aborted=false, @_request=#<ActionController::TestRequest:0x2aaab3203810 @request_uri="/", @env={"SERVER_NAME"=>"example.org", "HTTP_USER_AGENT"=>"Rails Testing", "rack.run_once"=>false, "rack.input"=>#<StringIO:0x2aaab31dfca8>, "PATH_INFO"=>"/", "rack.url_scheme"=>"http", "CONTENT_LENGTH"=>"0", "action_controller.request.request_parameters"=>{}, "rack.errors"=>#<StringIO:0x2aaab31dfd48>, "SCRIPT_NAME"=>"", "REMOTE_ADDR"=>"0.0.0.0", "rack.multithread"=>true, "rack.version"=>[0, 1], "rack.routing_args"=>{}, "rack.multiprocess"=>true, "SERVER_PORT"=>80, "QUERY_STRING"=>"", "REQUEST_METHOD"=>"GET"}, @cookies={}, @host="test.host", @parameters={}, @session={}, @request_method=:get, @content_type=nil, @query_parameters={}>>, @output_buffer=nil, @_current_render=nil, @assigns_added=true, @_first_render=#<ActionView::Template:0x2aaab3343838 @_memoized_method_name_without_locals=["_run_public4750046html"], @load_path="", @filename="/home/onecause/onecause/public/500.html", @format="html", @template_path="/home/onecause/onecause/public/500.html", @_memoized_method_segment=["public4750046html"], @name="500", @_memoized_relative_path=["public/500.html"], @extension=nil, @locale=nil, @base_path="/home/onecause/onecause/public", @_memoized_mime_type=[#<Mime::Type:0x2aaaad120c00 @symbol=:html, @string="text/html", @synonyms=["application/xhtml+xml"]>]>, @template=#<ActionView::Base:0x2aaab3389658 ...>, @template_format=:html, @assigns={}, @view_paths=["/home/onecause/onecause/app/views"], @helpers=#<ActionView::Base::ProxyModule:0x2aaab3389388>, @_request=#<ActionController::TestRequest:0x2aaab3203810 @request_uri="/", @env={"SERVER_NAME"=>"example.org", "HTTP_USER_AGENT"=>"Rails Testing", "rack.run_once"=>false, "rack.input"=>#<StringIO:0x2aaab31dfca8>, "PATH_INFO"=>"/", "rack.url_scheme"=>"http", "CONTENT_LENGTH"=>"0", "action_controller.request.request_parameters"=>{}, "rack.errors"=>#<StringIO:0x2aaab31dfd48>, "SCRIPT_NAME"=>"", "REMOTE_ADDR"=>"0.0.0.0", "rack.multithread"=>true, "rack.version"=>[0, 1], "rack.routing_args"=>{}, "rack.multiprocess"=>true, "SERVER_PORT"=>80, "QUERY_STRING"=>"", "REQUEST_METHOD"=>"GET"}, @cookies={}, @host="test.host", @parameters={}, @session={}, @request_method=:get, @content_type=nil, @query_parameters={}>>, @assigns=[], @writer=#<Proc:0x00002aaab322d958@/opt/ruby-enterprise-1.8.6-20080810/lib/ruby/gems/1.8/gems/actionpack-2.3.2/lib/action_controller/response.rb:46>, @header={"Content-Type"=>"text/html; charset=utf-8", "Cache-Control"=>"no-cache", "Set-Cookie"=>[], "Content-Length"=>"6367"}, @status="500 Internal Server Error", @block=nil>
Error 500. I ran it about 10 times, so perhaps you'll see it as an error in the Hoptoad app.
I'm eager to find a way to make this work, so if I can provide any information to help debug, just let me know.
Sean
Comments are currently closed for this discussion. You can start a new one.
2 Posted by Sean Hussey on 18 Aug, 2009 04:05 PM
I had our network vendor check the traffic between our app and hoptoad.com. Here are two traces (first is http, second is https):
3 Posted by Sean Hussey on 18 Aug, 2009 04:16 PM
Further, when running the send_to_hoptoad chain in the console, the response comes back like this:
When I run the entire set of code in the hoptoad:test task in the console, I get a 500 response from OUR app, not hoptoad. But the tcpdumps above show that the machines are talking to each other.
Support Staff 4 Posted by crowdtap on 21 Aug, 2009 08:32 PM
Sean,
We're talking about this and not coming up with much. This seems specific to your application as we haven't heard similar reports from anyone else.
That "peer certificate won't be verified in this SSL session" is odd. Why is SSL coming into play? Is your application using SSL? Your Hoptoad account is not using SSL.
Also, it is weird that ActionController::TestResponse is raised in production. You should be getting an HTTP response. Is there an environmental configuration somewhere in production that makes it think it is in test mode?
Support Staff 5 Posted by Joe Ferris on 21 Aug, 2009 08:58 PM
Hey Sean,
The response you see when running that code in the console is actually a response from your application code, in which we simulate a 500 error to trigger a Hoptoad notification. If you look closely at the response you pasted, you'll see that the layout is for "onecause.com," which I assume is the application that you're trying to setup with Hoptoad.
Can you try running this from your RAILS_ROOT on your server?
It should print a Net::HTTP response.
Thanks,
-Joe
6 Posted by Sean Hussey on 21 Aug, 2009 09:41 PM
Actually, before we get to that, did you tweak anything? I just received 2 valid exceptions from production. One before I received Dan's reply and one after Joe's reply.
Thanks!
Sean
7 Posted by Sean Hussey on 25 Aug, 2009 02:16 PM
Yup, I don't know what changed--we didn't change the app, I'm 99% sure the server and network configs didn't change, and I don't know what, if anything, you guys changed, but everything seems to be working fine now.
Any insight on your end?
Support Staff 8 Posted by Joe Ferris on 25 Aug, 2009 09:33 PM
Hey Sean,
I don't believe anything should have changed on our end that should affect your application. It's possible that there was some kind of network or caching issue between your application and our server.
Anyway, I'm glad to hear everything is working now!
-Joe
Joe Ferris resolved this discussion on 25 Aug, 2009 09:33 PM.
Sean Hussey re-opened this discussion on 26 Aug, 2009 01:39 AM
9 Posted by Sean Hussey on 26 Aug, 2009 01:39 AM
I guess it's got to be something in the middle, but nobody's copping to it. We've tried to get this to work on and off for more than a year, but no dice. Then we finally get access to the network admins and it starts working.
Well, I won't question. It works. We upgraded. Very happy.
Thanks!
Sean