MASTER firefox crash [@NP_Shutdown] [@Flash_EnforceLocalSecurity] -- libflashplayer.so

Flash player seems to be more crash-prone than usual on OSX. Reports usually indicate heavy flash usage (lots of tabs or intensive video sites) and only semi-reproducibly so this may be annoying to track down.

The three crash signatures listed in the summary are the three topcrashes for OSX and I think they're the same issue.

Cww, is this only for 3.0 or also for 3.5? Do you have some crash stats lists I can check for those three stacks? As Samuel pointed out yesterday it could be a dupe of bug 374630. But lets investigate first.

Bug 494448 covers [CFReadStreamGetStatus] and includes crashes on both 3.0.x and 3.5.

Basically go here: http://crash-stats.mozilla.com/query/query?do_query=1&product=Firefox&version=Firefox%3A3.0&version=Firefox%3A3.0.11&version=Firefox%3A3.5&version=Firefox%3A3.5b99&version=Firefox%3A3.5pre&platform=mac&date=&range_value=1&range_unit=weeks&query_search=signature&query_type=exact&query=

Then take the top 3 crashes and a few crashes below.

http://crash-stats.mozilla.com/report/list?product=Firefox&version=Firefox%3A3.0&version=Firefox%3A3.5&platform=mac&query_search=signature&query_type=exact&query=&date=&range_value=1&range_unit=weeks&do_query=1&signature=Flash_EnforceLocalSecurity

and

http://crash-stats.mozilla.com/report/list?product=Firefox&version=Firefox%3A3.0&version=Firefox%3A3.5&platform=mac&query_search=signature&query_type=exact&query=&date=&range_value=1&range_unit=weeks&do_query=1&signature=CFReadStreamGetStatus

and

http://crash-stats.mozilla.com/report/list?product=Firefox&version=Firefox%3A3.0&version=Firefox%3A3.5&platform=mac&query_search=signature&query_type=exact&query=&date=&range_value=1&range_unit=weeks&do_query=1&signature=Flash%20Player%400x91bd0

But also

http://crash-stats.mozilla.com/report/list?product=Firefox&version=Firefox%3A3.0&version=Firefox%3A3.5&platform=mac&query_search=signature&query_type=exact&query=&date=&range_value=1&range_unit=weeks&do_query=1&signature=Flash%20Player%400x85e6b

and

http://crash-stats.mozilla.com/report/list?product=Firefox&version=Firefox%3A3.0&version=Firefox%3A3.5&platform=mac&query_search=signature&query_type=exact&query=&date=&range_value=1&range_unit=weeks&do_query=1&signature=Flash%20Player%400x2dd943

and

http://crash-stats.mozilla.com/report/list?product=Firefox&version=Firefox%3A3.0&version=Firefox%3A3.5&platform=mac&query_search=signature&query_type=exact&query=&date=&range_value=1&range_unit=weeks&do_query=1&signature=Flash%20Player%400x91ac4

I really, really, really don't know why this would happen, but this is the #1 topcrash on Mac *by far*. In the last week we've seen ~2,200 crashes of the previous topcrash. In comparison, Flash topcrashes total over 22,000 in the last week. 10 times as many!

Michelle: I'm not sure if you're the right person to give this to, but can you please direct this to the appropriate person? We're seeing far more Mac crashes in Flash than we've seen in a long time. Is this something you're seeing on your end? Is it something being tracked? Feel free to email me (<email address hidden>) if you want to take this offline.

In the meantime, for support purposes, I'd recommend Flashblock to users who are experiencing an influx of Flash crashes. There's a good chance a major site (YouTube, some ad site, etc) changed and that code is causing this issue. We should get a list of URLs this happens on and sort it. (I'm on it!)

I filed bug 500582 to get URLs for these crashes. Note that it's confidential for privacy reasons.

bug 500574 has details on potentially some of this.

(In reply to comment #6)
> bug 500574 has details on potentially some of this.

Could you cc: me on bug 500574? Thanks!

When I look at the CFReadStreamGetStatus crash in mozilla crash-stats, the graph shows that this crash peaked from May 19th through June 5th. It had another mini-peak later from June 22nd through June 25th. Is this graph correct? Is this crash no longer happening nearly as much? I wonder if a specific piece of content (such as an ad) came out that exhibited this crash, and then that content was pulled around June 5th (ignoring the second peak for now).

Another question: in mozilla crash-stats, is the Flash Player version (e.g, 10.0.12, 10.0.22, etc.) stored anywhere? I don't see it in the raw dump or anywhere else.

Do you have a reproducible test case? We are trying lots of the listed URLs but haven't crashed.

(In reply to comment #8)
> Another question: in mozilla crash-stats, is the Flash Player version (e.g,
> 10.0.12, 10.0.22, etc.) stored anywhere? I don't see it in the raw dump or
> anywhere else.

Michelle, are you searching for something like that?

NPSWF32.dll 10.0.22.87 3BB359BF2B484134883563A2C8D4860C1 NPSWF32.pdb

This entry you should be able to find under loaded modules. The second column holds the version of the installed Flash player.

Unfortunately, under the "Modules" tab (the other tabs are "Details" and "Raw Dump"), I only see this:

Flash Player 860692A215F054B7B9474B410ABEB5300 Flash Player

Where are you finding the NPSWF32.dll and version data? Thanks!

(In reply to comment #10)
> Unfortunately, under the "Modules" tab (the other tabs are "Details" and "Raw
> Dump"), I only see this:
>
> Flash Player 860692A215F054B7B9474B410ABEB5300 Flash Player
>
> Where are you finding the NPSWF32.dll and version data? Thanks!

He is looking at Window's crashes which do report the version.

(In reply to comment #10)
> Unfortunately, under the "Modules" tab (the other tabs are "Details" and "Raw
> Dump"), I only see this:
>
> Flash Player 860692A215F054B7B9474B410ABEB5300 Flash Player
>
> Where are you finding the NPSWF32.dll and version data? Thanks!

Sorry Michelle. Yes I was looking for crashes on Windows. But I wonder why we don't have version information available on OS X. Ted, do you have an idea why?

I think Breakpad just doesn't support that on OS X. (The concept of version numbers for binaries isn't as straight forward on OS X as it is on Windows, IIRC.)

See also http://code.google.com/p/google-breakpad/issues/detail?id=321

Note that the "Debug Identifier" on OS X is essentially an MD5 sum of part of the binary. If you were to run the dump_syms binary on a library, the first line of the output would be a MODULE line including the debug identifier as calculated for that binary. The dump_syms tool can be built by checking out the Breakpad source:
http://code.google.com/p/google-breakpad/source/checkout
and opening the XCode project under src/client/mac.

The crashes at CFReadStreamGetStatus all have the same stack, I think. We have fixed that crash in-house and it will be in an upcoming Flash Player release.

The other two crashes (Flash_EnforceLocalSecurity and Flash Player@0x91bd0) actually represent a bunch of different crashes; you can see they have many different stacks, unfortunately.

Michelle: if Adobe is interested, and has unstripped binaries of the shipping Flash Player plugin available, we could get the symbols installed on our symbol server such that these crash reports would have actual function symbols. Currently the binary is stripped, so if we have any symbols they're just the declared entry points. (Which is why things like Flash_EnableLocalSecurity show up in stacks, despite them being unlikely to be the culprit.)

We don't need full debug symbols, just function-level symbols would give you realistic stacks here. Feel free to contact me here or via e-mail if you're interested.

Ted, would the debug version already help which can be downloaded from this location? http://kb2.adobe.com/cps/142/tn_14266.html

Those don't appear to have actual debug symbols, I think they have some use for content creators, but not for us.

We indeed have confirmed this particular crash at CFReadStreamGetStatus fixed internally. It will ship with our next release.

Could you please point us to a reproducible crash at Flash Player@0x91bd0? Thank you so much!

Flash 10.0.32.18 just came out, hope this fixes it:
http://get.adobe.com/flashplayer/

The Flash Player team released a new version (10.0.32.18) on Friday that fixes this crash. We'd appreciate any feedback before closing this bug report. Thanks!

Thanks Charles! We should wait a week or better two to get the results. Depending on the update ratio it can take a while until we can see a change.

*** Bug 374630 has been marked as a duplicate of this bug. ***

Ubuntu Bug:
https://bugs.launchpad.net/bugs/69931

Guys how are the stats looking on your end when it comes to this crash. Its hard to tell where things are with this from the limited searching I can do.

Charles: http://tinyurl.com/kporzy shows a total of 5947 crashes in CFReadStreamGetStatus using Firefox 3.5.2. But it is hard to tell which version of flash these people are using.

(In reply to comment #26)
> Guys how are the stats looking on your end when it comes to this crash. Its
> hard to tell where things are with this from the limited searching I can do.

Flash_EnforceLocalSecurity is gone due to the removal of the bad Flash symbols on Socorro. CFReadStreamGetStatus has increased by 25%. Flash Player 0x91bd0 is pretty constant.

Flash users just don't upgrade.

i get the following alert which renders firefox unusable:

"a connection to the debugger or profiler could not be estableished by adobe flash player 9."

i'm running your latest firefox on an imac running snow leopard.

>i get the following alert which renders firefox unusable:
>"a connection to the debugger or profiler could not be estableished by adobe flash player 9."

This is unrelated to this bug. I've emailed the poster offline about their problem.

The report at http://people.mozilla.org/~dbaron/crash-stats/20090929-interesting-addons-with-versions (linked from http://dbaron.org/mozilla/topcrash-modules) now has the library checksums (or whatever the thing for linking to debug symbols is) which Sam has compiled data on at https://wiki.mozilla.org/CrashKill/Mac_Flash_Identifiers .

So, for example, it shows:
  CFReadStreamGetStatus|EXC_BAD_ACCESS / KERN_PROTECTION_FAILURE (1744 crashes)
...
    100% (1744/1744) vs. 90% (16597/18510) Flash Player
...
          0% (0/1744) vs. 7% (1343/18510) checksum: 025105C956638D665850591768FB743D0
...
          2% (37/1744) vs. 2% (280/18510) checksum: 38AEB67F6A0B43C6A341D7936603E84A0
...
          6% (112/1744) vs. 4% (791/18510) checksum: 77CB5AC61C456B965D0B41361B3F6CEA0
...
         76% (1327/1744) vs. 23% (4198/18510) checksum: 860692A215F054B7B9474B410ABEB5300
...
         10% (172/1744) vs. 2% (379/18510) checksum: 986682965B43DFA62E0A0DFFD7B7417F0
...
          0% (0/1744) vs. 43% (7924/18510) checksum: B19EE2363941C9582E040B99BB5E237A0

So your data set shows zero crashes with 10.0.32.18 (the latest version). The majority of the crashes with that signature were using 10.0.22.87, which is one version out of date (not counting the version that shipped with 10.6).

Great to hear conformation. Feel free to close this bug at any time.

this crash is occurring in 3.6b5 with Flash player v. 10.0.32.18. repeatedly verifiable for me on two machines with the same application versions by visiting this link: http://www.speaklight.com/design, and clicking the play button on the video (or manually triggering the .click() event on that img via the console).

Flash 10.0.32.18 is out of date. The most recent version is 10.0.42.34.

hey Bob, thanks for the heads up. It still happens with 10.0.42.34, however after more poking around i think this is a flash issue, not a FF issue. retracted!

Ian, thanks. I tried on mac and windows. The movie doesn't load. No crash. Not sure what is up. If you can submit a crash report and paste the id here that would be great. What OS, version? Are you sure the Flash plugin that is loading is the most current? Does Addons->Plugins agree?

The situation deteriorated on my machine over the course of the day to the point that I was seeing immediate crashes upon loading any flash content in FF, Chrome Mac Beta and Safari 4. I uninstalled Flash from my system, and did some general housekeeping and have not been able to reproduce the problem since. FF crash reporter was not triggered on any of the multiple crashes I experienced yesterday, the most recent I have listed in about:crashes is 6582669b-1ded-46ab-9e7a-a541d2100103, which is also a Flash issue from a couple of days ago, not sure if it's related or not.

This is a case where if you right click on the movie Flash Player will tell you it is not loaded. We prevented the loading of the movie because of an error in the movie such as a path problem or bad data.

There is no movie not loaded issue for me when testing this on Flash Player 10.1 beta 2.

Guys, it's clear from comment 31/comment 32 that the flash team did manage to fix something here.

i trust that they've fixed something and would like to encourage everyone to limit the lifespan of a single bug to a single issue, once it's fixed, we encourage people to file a new bug, even if the steps to reproduce are the same. it makes it easier to track things this way.

Version and milestone values are being reset to defaults as part of product refactoring.