Skip to main content
Home Forums General Silverlight New Features in Silverlight 3 WriteableBitmap with MediaElement
9 replies. Latest Post by DarkGold on November 5, 2009.
(0)
brucechase
Member
78 points
31 Posts
08-07-2009 12:50 PM |
I have found a problem with the a great feature -- WriteableBitmap. It may be that the new functionality of WriteableBitmap may exclude the MediaElement content from having the pixel bits accessed or changed.
Essentially, I use WriteableBitmap to capture a series of images from frames of a playing video. The images are captured with no problem at all. Matter of fact, I put the captured images back onto the screen, one on top of another with differing Z values and opacity such that one can see the object of motion that occurs over time from the video in what now appears to be a single still image. But I want to do more. I would like to set individual pixel opacity and color. Upon attempting this, I received an error:
Could not read pixel System.Security.SecurityException: "WriteableBitmap has protected content. Pixel access is not allowed."
at System.Windows.Media.Imaging.WriteableBitmap.get_Pixels()
I am grabbing the videos from the same domain as the website -- WebDev's 127.0.0.1.
From looking at the problem, it appears that if a MediaElement is contained within the visual tree descendants of any UIElement that is used as a parameter to WriteableBitmap, the individual pixels cannot be accessed. In my case, I was doing this:
{
}
Moe Elshall
188 points
29 Posts
08-07-2009 1:06 PM |
WriteableBitmap has some built-in security features. Capturing a visual tree that contains any cross domain content (image, video,...etc) into a WriteableBitmap and attempting to read/write individual pixels would result in the exception you're getting here (SecurityException: WriteableBitmap has protected content. Pixel access is not allowed).
You mentioned that your video is on the same domain, in that case, you should be able to access the pixels.
brucechase:From looking at the problem, it appears that if a MediaElement is contained within the visual tree descendants of any UIElement that is used as a parameter to WriteableBitmap, the individual pixels cannot be accessed.
When it comes to pixel access, whether you render the MediaElement itself or any of its ancestors shouldn't matter.
If you're able to get to the pixels when you render the MediaElement but getting a SecurityException when rendering one of its ancestors, then we have a bug. _unless_ somewhere in the tree, there is some other cross-domain content!
Would you please give us some more detail?
08-07-2009 1:45 PM |
The video is definitely from the local machine for which VS WebDev is hosting the site in debug mode. The MediaElement' source from the debugger:
http://localhost:2237/videos/8d04fa49-ecf6-46d0-880c-51ce04af4742.wmv
Again, the video plays fine; the WriteableBitmap captures a video frame fine; and the captured video frame, as an image displays fine. It is when I attempt to access the bits (in particular, the algorithm was looking for differences -- motion -- between frame shots), that I get the error. If I make a WriteableBitmap with a UIElement in the same visual tree for which the MediaElement is not a descendant, I can access the bits just fine.
I attempted, for testing purposes, to change the MediaElements visibility to Visibility.Collapsed. The resulting images from WriteableBitmap, using the MediaElement as a source paramenter, were black (make sense). Their bits could not be accessed either.
Next, for testing, I used a parent of the MediaElement -- with the MediaElement visibility set to Visibility.Collapsed -- and I could not acces the pixels again even though the MediaElement was collapsed I was now using different UIElement. From this, I concluded that any UIElement used as a source paramenter for WriteableBitmap, that has a child descendant of type MediaElement, would create bitmap bits that could not be accessed for security reasons.
I put the control onto our site so that you can view the results of WriteableBitmap in action -- but without updating the bits for transparency. The control will do a rudamentary job of capturing "motion" in realtime as the video is playing and display the result. As you can see, the WriteableBitmap does a great job of capturing 10 images over a few seconds and then the images are set on top of one another with varying opacity. Go to:
http://www.CoachMeSports.com?xamlID=55d766d0-aaca-4172-8e00-1aea8c7da5b0
to see it in action.
I hope this helps.
Bruce
08-07-2009 4:35 PM |
Moe,
I have tracked down the problem for the Silverlight team. The Uri address is very particular for the video in order to have pixel access via the WriteableBitmap function. My Silverlight application and video sits on the CoachMeSports.com server. All addresses below will pull up the same application.
If I start up the application as with: http://coachmesports.com in the address bar, the pixel access is denied.
If I start up the app with: http://www.coachmesports.com in the address bar, the Silverlight program can access the pixels.
If I start up the app with: http://216.205.99.21 in the address bar, the pixel access is denied....
BUT if I change the Uri of the video within the code to go to:
http://216.205.99.21 (with all relevant virtual directories), the access is allowed.
So, the video Uri must match exactly the address bar in order to work even though the Silverlight application works fine from all the above address since they point to exactly the same asp page.
I am sure there is some relevant factors involved that can make more code more responsible...but it is nice to know I can get this to work for most people as long as they put www in the address. Often, I do not.
I have not tried changing case. Hopefully it is not case sensitive.
08-07-2009 5:56 PM |
That's what I expected.
The domain name resolver in silverlight (that's used by WriteableBitmap) requires that the website domain and that of the MediaElement to be same to allow pixel access. And it's not case sensitive (so, no worries :) )
You've mentioned earlier that changing Visibility (Setting the MediaElement to Collapsed) didn't change the WriteableBitmap pixel access behavior, that's by design. When it comes to security, WriteableBitmap is very aggressive. As long as the source is cross domain and even if the element is Collapsed, pixel access will be completely blocked. We decided to err on the side of security.
08-07-2009 7:22 PM |
Have you tried this then with a site that is located at www.yourname.com for which you might retrieve videos at mms://yourname.com/... when using Windows Media Server? I presume from the above experiment that it will not work.
Thanks for the quick reply.
zleek
4 points
2 Posts
11-03-2009 10:22 AM |
Ouch... this one just bit me today. My production site uses subdomains to serve content but dev/test does not (example: SL app hosted on zleek.com, content from img1.zleek.com, img2.zleek.com, etc.).
We were also planning on moving video content to a streaming service. Because of this, we won't be able to do that and use WriteableBitmap for snapshots? Can writeableBitmap be changed to work with crossdomain.xml/clientaccesspolicy.xml?
BTW, Bruce, hope to be with you in the top 5 at ComponentArt this afternoon. Good luck! :)
11-03-2009 11:16 AM |
Good luck to you as well. Your application is well done.
The writeable bitmap would be a quick way of building a preview image of an album page that contains videos for you. Since my application uses WriteableBitmap so much, my videos come from a single server. I have not delved into any method lately to overcome this.
I noticed your Facebook interface. That took some work.
11-03-2009 2:55 PM |
I'm using WriteableBitmap in two places...
1. Drag/Drop operations for reordering pages. Quick and easy way for providing a drag/drop preview instead of having to inspect the visual tree and recreate it. This works fine as I'm not accessing individual pixels.
2. Exporting pages. I wanted to give the users a way to export their layouts to an image. Eventually, this may turn into a feature similar to some photo sites out there that let you upload pictures and they'll send you a printed book in the mail. So what I'm doing here is using WriteableBitmap to get a capture of the current page, then read the individual pixels and send them through a JPEG encoder -- that's where I'm having the issue...
Oh well, I guess I can host the media on the same domain. The reason I split it to subdomains was to get around browser connection restrictions (2 in IE7, 6 in Firefox/IE8). That way downloading an image wouldn't block a web service call from happening.
BTW, that motion control capturing the diffs between frames is slick!
DarkGold
2 points
4 Posts
11-05-2009 5:53 AM |
Moe Elshall:That's what I expected. The domain name resolver in silverlight (that's used by WriteableBitmap) requires that the website domain and that of the MediaElement to be same to allow pixel access. And it's not case sensitive (so, no worries :) ) You've mentioned earlier that changing Visibility (Setting the MediaElement to Collapsed) didn't change the WriteableBitmap pixel access behavior, that's by design. When it comes to security, WriteableBitmap is very aggressive. As long as the source is cross domain and even if the element is Collapsed, pixel access will be completely blocked. We decided to err on the side of security.
Hi all,
I'm facing a similar problem, I have a publishing point (webcam) on a WMS server (mySvr) coming from a webcam encoded live by EE3.
I want to acquire/modify a picture frame of the displayed video so in my SL3 application I use a WriteableBitmap on the MediaElement.
To acquire the video I must use the "mms://mySvr:8080/webcam" scheme (http fallback), everything works fine, except when the writeableBitmap.Pixels is called, I get a SecurityException !
My understanding is that my application domain name is "mySvr" and my video domain name is "mySvr:8080", so there is a cross domain security exception, is this right ?
What can I do to solve this problem with "live" content ? Using a file "http://mySvr/test.wmv" everything is ok ....
WBR.
WriteableBitmap with MediaElement