--- /dev/null
-
- void gtk_widget_size_allocate (GtkWidget *widget,
- GtkAllocation *allocation);
-
+* Xwidgets
+
+This is an experimental branch to enable embedding of GTK widgets
+inside an Emacs window. The Emacs abstraction is called an Xwidget,
+for eXternal widget, and also in reference to the Xembed protocoll.
+
+There is a demo file called xwidget-test.el which shows some of the
+possibilities. There are some screnshots at the emacswiki.
+
+Currently its possible to insert buttons, sliders, xembed widgets, and
+webkit in the buffer. It works similar to the support for images in
+Emacs. Adding more types of widgets should be fairly straightforward,
+but will require adapter code for each type.
+
+A difference from images is that xwidgets live their own life. You
+create them with an api, get a reference, and tie them to a particular
+buffer with a display spec.
+
+Each xwidget can have several views. In MVC terms, an xwidget is the
+model, and an xwidget-view is a view of the xwidget in a particular
+Emacs window.
+
+The xwidget code attempts to keep the visual appearance of the views
+in sync with through an Observer pattern implementation. This is
+necessary to support the Emacs window paradigm.
+
+** building
+bzr co bzr+ssh://bzr.savannah.gnu.org/emacs/xwidget/
+#the below compiler flags shouldn't be strictly necessary
+export CFLAGS=" -g"
+./configure --with-xwidgets --enable-asserts --with-x-toolkit=gtk3
+make -j4
+gdb -ex run src/emacs
+
+** try it out
+If you have GTK3 and gtk-webkit installed, you should be able to
+start the embedded webkit browser now:
+
+M-X xwidget-webkit-browse-url
+
+If that didnt work out try the minimal demonstration instead:
+
+(load-library "xwidget-test")
+(xwidget-demo-a-button)
+
+It looks unimpressive, but it's a gtk button inside an Emacs buffer!
+*** webkit hints
+If you got webkit working, great! Please note, though, that the
+current support is far from a full fledged browser. My focus is on
+delivering a component that can be used to build a full emacs based
+browser on. Since I implement a browse-url function you can get quite
+far just by:
+
+(setq browse-url-browser-function 'xwidget-webkit-browse-url)
+
+then all Emacs browser interface systems work to a degree.
+heres stuff I use currenly
+
+- m-x anything-surfraw interfaces to search engines
+- C-o in org mode opens links inside org
+- m-x ffap opens links anywhere in a buffer
+- m-x gtk-lookup-symbol searches gtk docs
+- m-x bookmark-jump
+etc..
+
+I'll add more examples as I go along.
+
+However theres lots of support missing, see TODO list for
+information. Briefly:
+- download handling
+- keyboard field navigation
+- isearch
+- history
+- sites that use flash. I dont really care about this issue so its
+unlikely to be fixed. Just a heads up.
+
+** Stability
+Beginning with Summer 2011 I am now able to use Xwidget Emacs as my
+primary Emacs. That is good for the project and the stability of the
+code.
+
+At the time of writing I have 24 hour Emacs uptime with several
+embedded webkit browsers, Gnus, org-mode, tramp, etc. etc.
+
+That said, there are still many improvements that needs to be done,
+particularily in memory management. Expect xwidget emacs to leak
+heavily for now.
+
+** timeline for inclusion in trunk
+The Emacs 24 feature freeze is passed, so xwidgets won't probably be merged
+until Emacs 25. OTOH since I now use xwidget emacs as my primary
+emacs, I will merge from trunk much more often than in the past.
+
+** Thanks
+emacs-devel@gnu.org. There are very helpful people there. When I
+started the xwidget project I had no clue about the Emacs internals.
+
+* Brief overview of how xwidgets work
+Xwidgets work in one way like images in Emacs. You bind a display spec very
+similar to an image display spec to buffer contents. The display engine will
+notice the display spec and try to display the xwidget there. The display engine
+prepares space at the right place for the xwidget and so on for free, as long as
+we provide proper sizes and so on back to the redisplay engine.
+
+** Issues
+The problem is that Emacs cant actually draw the widgets, as it can with
+images. Emacs must notify GTK about where the widgets should be, and how they
+should be clipped and so on, and this information must be given to GTK
+synchonous with Emacs display changes. Ok, so why is that difficult then?
+
+- How do we know when a widget is NOT to be drawn? The only way I found so far
+ is having a flag for each xwdiget, that is reset before a redisplay. When an
+ xwidget is encountered during display, the flag is set. After redisplay,
+ iterate all xwidgets and hide those which hasnt been displayed.
+
+- The gtk socket type for embedding external applications is desirable
+ but presents a lot of difficulties of its own. One difficulty is
+ deciding which input events to forward, and when and how to do it.
+
+** placement and clipping
+the entire emacs frame is a gtk window. we use the fixed layout
+manager to place xwidgets on the frame. coordinates are supplied by
+the emacs display engine. widgets are placed inside an intermediate
+window, called the widgetwindow. the widgetwindows are placed on the
+emacs frame.
+
+this way was chosen to simplify clipping of the widgets against emacs
+window borders.
+
+
+** different strategies
+Integrating toolkit widgets(gtk in this case) and the emacs display
+engine is more difficult than your plain average gui application, and
+different strategies has been tested and will continue to be tested.
+
+There was a distinction between live xwidgets and
+phantom xwidgets, previous to the change to MVC.
+
+- the first aproach was to have the live xwidget on-screen, and move
+ them about. the phantoms were generated by snapshoting the live
+ xwidget.
+
+the drawback of that aproach was that the gtk toolkit is admirably
+lazy and doesnt draw the widget if its not actualy shown, meaning that
+the snapshots for the phantoms will show garbage.
+
+- the second aproach was to use composition support. that tells gtk
+ that the widget should be drawn in an off-screen buffer and drawn on
+ screen by the application.
+
+this has the primary advantage that the snapshot is always
+available, and enables the possibility of more eye-candy like drawing
+live and phantom widgets in different colors.
+
+the drawback is that its our own responsibility to handle drawing,
+which puts more of the display optimization burden on us.
+
+this is aproach worked so-so.
+
+- another aproach is to have both live and phantom widgets drawn
+ on-screen by proxy gtk objects. the live xwidget will be entirely
+ handled in an off-screen window, and the proxy objects will redirect
+ events there.
+
+- combine on-screen and off-screen aproaches. maybe composition is the
+ way to go for most cases, but on-screen xembeding is the way to go
+ for particular special cases, like showing video in a
+ window. off-screen rendering and whatnot, is not efficient in that
+ particular case, and the user will simply have to accept that the
+ phantom of a video widget isnt particularily beautiful.
+
+- The current and seemingly sanest aproach implements a MVC pattern.
+
+** Testing
+;;test like:
+;; cd /path/to/xwidgets-emacs-dir
+;; make all&& src/emacs -q --eval "(progn (load \"`pwd`/lisp/xwidget-test.el\") (xwidget-demo-basic))"
+
+** MVC and Xembedd
+The MVC approach appears to be at least in principle robust for plain gtk
+widgets. For the interesting case of gtk sockets which implements an
+xembed host widget that allows for embedding other applications inside
+an Emacs window, the story gets more complex.
+
+The problem is that xembed is designed to plug an application window
+inside a a secket and thats it. You can't move a plug between
+sockets. I tried numerous hacks to get around this but there is
+nothing that works realy well.
+
+Therefore the Emacs part of the code will only expose well-defined
+interfaces. cooperating applications will be able to use the interface
+in a well defined manner. The problem is that there is no known xembeddable
+application that implement the needed type of functionality, which is
+allowing for creating new windows on the fly that plug into new
+sockets.
+
+Therefore I will attempt to provide an external application that wraps
+another application and through hacks attempts to provide the needed
+multi view xembed function. That way Emacs is sane and the insanity
+contained.
+
+This app will work by providing a socket that an app plugs into. The
+socket window is copied efficientlp by means of composition to a
+number of other windows, that then are plugged into the different
+Emacs sockets.
+** old notes from x_draw_xwidget_glyph_string
+
+ BUG it seems this method for some reason is called with bad s->x and s->y sometimes.
+ When this happens the xwidget doesnt move on screen as it should.
+ This mightbe because of x_scroll_run. Emacs decides to scroll the screen by blitting sometimes.
+ then emacs doesnt try to actualy call the paint routines, which means this here code will never
+ run so the xwidget wont know it has been moved.
+
+ Solved temporarily by never optimizing in try_window_reusing_current_matrix().
+
+ BUG the phantoming code doesnt work very well when the live xwidget is off screen.
+ you will get weirdo display artefacts. Composition ought to solve this, since that means the live window is
+ always available in an off-screen buffer. My current attempt at composition doesnt work properly however.
+
+ //allocation debugging. the correct values cant be expected to show upp immediately, but eventually they should get to be ok
+ // this is because we dont know when the container gets around to doing layout
+ //GtkAllocation galloc;
+ //gtk_widget_get_allocation(GTK_WIDGET (xv->widgetwindow), &galloc);
+ //printf("allocation %d %d , %d %d\n", galloc.x,galloc.y,galloc.width,galloc.height);
+
+
+*** old notes about the old live/phantom scheme
+
+ //TODO:
+ // 1) always draw live xwidget in slected window
+ // (2) if there were no live instances of the xwidget in selected window, also draw it live)
+ // 3) if there was a live xwidget previously, now phantom it.
+
+ else
+ {
+ //ok, we are painting the xwidgets in non-selected window, so draw a phantom
+ //printf("draw phantom xwidget at:%d %d\n",x,y);
+ //xwidget_composite_draw_phantom (xw, x, y, clipx, clipy); //TODO MVC there will be very few cases of phantoming
+ }
+
+
+ atm this works as follows: only check if xwidgets are displayed in the
+ "selected window". if not, hide them or phantom them.
+
+ this means valid cases like xwidgets being displayed only once in
+ non-selected windows, does not work well. they should also be visible
+ in that case not phantomed.
+
+* ToDo:s
+** DONE new annoying trace
+ CLOSED: [2011-08-13 Sat 16:16]
+maybe related to scroll inhibiting or cursor inhibiting code.
+It appears actually to be related to GLYPH_DEBUG=1. this flag is no
+longer needed.
+*** the trace
+Breakpoint 1, abort () at emacs.c:383
+383 kill (getpid (), SIGABRT);
+Missing separate debuginfos, use: debuginfo-install hunspell-1.2.15-2.fc15.x86_64 nss-mdns-0.10-9.fc15.x86_64
+(gdb)
+(gdb)
+(gdb) bt
+#0 abort () at emacs.c:383
+#1 0x0000000000418f01 in matrix_row (matrix=0xac29400, row=-1)
+ at dispnew.c:1477
+#2 0x000000000046e113 in draw_glyphs (w=0x18235c0, x=198, row=0xa3af100, area=
+ TEXT_AREA, start=17, end=18, hl=DRAW_CURSOR, overlaps=0) at xdisp.c:22550
+#3 0x000000000047869f in draw_phys_cursor_glyph (w=0x18235c0, row=0xa3af100,
+ hl=DRAW_CURSOR) at xdisp.c:24882
+#4 0x00000000005083bb in x_draw_window_cursor (w=0x18235c0, glyph_row=
+ 0xa3af100, x=180, y=361, cursor_type=0, cursor_width=1, on_p=1, active_p=1)
+ at xterm.c:7440
+#5 0x00000000004790cd in display_and_set_cursor (w=0x18235c0, on=1, hpos=17,
+ vpos=19, x=180, y=361) at xdisp.c:25098
+#6 0x00000000004fa31f in x_update_window_end (w=0x18235c0, cursor_on_p=1,
+ mouse_face_overwritten_p=0) at xterm.c:644
+#7 0x000000000041ccb9 in update_window (w=0x18235c0, force_p=0)
+ at dispnew.c:3694
+#8 0x000000000041c165 in update_window_tree (w=0x18235c0, force_p=0)
+ at dispnew.c:3331
+#9 0x000000000041beee in update_frame (f=0x1658460, force_p=0,
+ inhibit_hairy_id_p=0) at dispnew.c:3258
+#10 0x0000000000450a2e in redisplay_internal () at xdisp.c:12983
+#11 0x000000000044e2a6 in redisplay () at xdisp.c:12099
+#12 0x000000000056a60d in read_char (commandflag=1, nmaps=6, maps=
+
+** DONE allow xwidgets to report their size
+ CLOSED: [2011-07-19 Tue 14:26]
+now we just hard code sizes. but webkit widgets for instance can
+report sizes that suit the content. support that.
+** TODO BUG xwidget view ghosts
+- xwidget-webkit-browse-url somewhere
+- split window.
+now theres 2 webkit views
+- c-x 1
+now theres 2 views but one is a ghost!
+one should have been deleted when its window died but that didnt work
+for some reason here.
+
+- m-x xwidget-cleanup
+
+the ghost goes away because we killed explicitly but this is just a workaround.
+
+xwidget_view_delete_all_in_window(w); in delete-window-internal is not sufficient.
+delete-other-windows-internal
+delete_all_subwindows
+unshow_buffer
+
+Added cleanup those window configuration hook which works in practice
+but feels kludgy.
+
+** DONE BUG annoying backtrace
+ CLOSED: [2011-07-19 Tue 14:28]
+(this no longer seems to happen even under heavy usage. seems merging
+from trunk helped. lots were happening in redisplay at this time in trunk.)
+
+sadly happens a lot.
+- happens even with no initialized xwidgets
+- + row->glyphs[area][i].face_id
+or similar code, so row is invalid for some reason.
+xwidgets currently disable some redisplay opimizations so it might be
+an actual emacs bug manifesting without optimizations.
+
+*** bt 1
+ /* Compute the width of this line. */
+ row->pixel_width = row->x;
+ for (i = 0; i < row->used[TEXT_AREA]; ++i)
+ row->pixel_width += row->glyphs[TEXT_AREA][i].pixel_width;
+
+(gdb) bt
+#0 0x000000000045c340 in compute_line_metrics (it=0x7fffffff8a20)
+ at xdisp.c:17549
+#1 0x00000000004603da in display_line (it=0x7fffffff8a20) at xdisp.c:18792
+#2 0x0000000000457646 in try_window (window=23403045, pos=..., flags=1)
+ at xdisp.c:15399
+#3 0x00000000004559c9 in redisplay_window (window=23403045, just_this_one_p=0)
+ at xdisp.c:14944
+#4 0x0000000000450247 in redisplay_window_0 (window=23403045) at xdisp.c:13152
+#5 0x00000000005fdcd9 in internal_condition_case_1 (bfun=
+ 0x450208 <redisplay_window_0>, arg=23403045, handlers=12691046, hfun=
+ 0x4501d9 <redisplay_window_error>) at eval.c:1538
+#6 0x00000000004501ba in redisplay_windows (window=23403045) at xdisp.c:13132
+#7 0x000000000044f19c in redisplay_internal () at xdisp.c:12706
+#8 0x000000000044f9f2 in redisplay_preserve_echo_area (from_where=7)
+ at xdisp.c:12964
+#9 0x0000000000568525 in swallow_events (do_display=1) at keyboard.c:4197
+#10 0x0000000000422554 in sit_for (timeout=40, reading=1, do_display=1)
+ at dispnew.c:5963
+#11 0x000000000056512c in read_char (commandflag=1, nmaps=8, maps=
+ 0x7fffffffd3f0, prev_event=12720514, used_mouse_menu=0x7fffffffd604,
+ end_time=0x0) at keyboard.c:2689
+#12 0x0000000000572c59 in read_key_sequence (keybuf=0x7fffffffd850, bufsize=
+ 30, prompt=12720514, dont_downcase_last=0, can_return_switch_frame=1,
+---Type <return> to continue, or q <return> to quit---
+ fix_current_buffer=1) at keyboard.c:9291
+#13 0x0000000000562897 in command_loop_1 () at keyboard.c:1446
+#14 0x00000000005fdb52 in internal_condition_case (bfun=
+ 0x5624b4 <command_loop_1>, handlers=12772898, hfun=0x561dab <cmd_error>)
+ at eval.c:1493
+#15 0x00000000005621ab in command_loop_2 (ignore=12720514) at keyboard.c:1157
+#16 0x00000000005fd4ce in internal_catch (tag=12768770, func=
+ 0x562185 <command_loop_2>, arg=12720514) at eval.c:1247
+#17 0x000000000056215e in command_loop () at keyboard.c:1136
+#18 0x00000000005618f9 in recursive_edit_1 () at keyboard.c:757
+#19 0x0000000000561a95 in Frecursive_edit () at keyboard.c:821
+#20 0x000000000055fba2 in main (argc=1, argv=0x7fffffffe188) at emacs.c:1704
+
+
+*** bt 2
+
+** DONE Examine using XComposite rather than GTK off-screen
+ rendering. This would make xembed widgets work much better. This
+ would probably be rathter difficult, but could open up other
+ interesting possibilities for Emacs. There is an early attempt in
+ xwidget.c, but the X call to redirect to offscreen rendering fails
+ for unknown reasons.
+
+ the attempt was further worked on, and the xlib calls replaced with
+ gdk calls, this works better.
+
+ In the end I abandoned this aproach. Xwidget-osr is the new aproach.
+
+** TODO make the keyboard event code propagation code work.
+There is an attempt to provide an api to send keyboard events to an
+xwidget, but it doesnt currently work very well.
+
+*** TODO try gtk event creation instead
+since that works fine in the webkit osr code.
+but, oh no, that didn't work for some reason.
+the widgets seems to receive the event but then the embedded widgets
+hangs.
+
+
+*** TODO examine some library to synthesise events
+xdotool
+xte xautomation
+crikey
+libxdo
+** DONE remove the special-case for when the minibuffer is
+ active. I added some code to reduce the annoying problem display artefacts
+ when making the minibuffer the selected window. This made xwidgets in the
+ buffer go grey or black whenever one did m-x to activate the minibuffer. The
+ coded tried to handle the minibuffer as a special case. That simply wasnt a
+ good idea. Special-casing will never work properly. It is much better to spend
+ time finding solutions that work acceptably in the general case.
+
+** DONE disable emacs cursor drawing on top of an active xwidget.
+ This ought to be rather simple and should improve the visuals a lot.
+
+** TODO improve the xwidgets programming interface
+so its less of hand-waving affair. This shouldnt be too hard, but I
+ have deliberatley not spent any time on it, since getting the
+ visuals right is much harder. Anyway, I sort of think the interface
+ should be somewhat like it is, except symbols is used instead of
+ integers.
+*** DONE use symbols for xwidget types rather than ints
+ CLOSED: [2011-06-27 Mon 12:52]
+
+
+*** TODO better lisp based structure for xwidgets
+the lisp interface woud be like this:
+- make-xwidget returns an xwidget object, similar to a process
+ object. this object is used when creating the display spec(instead of
+ the user defined id now used)
+
+the data structure would be something like this:
+- a "process" like aproach to create the xwidgets. xwidgets are
+ coupled to buffers, somewhat like processes, except a buffer can
+ hold several xwidgets
+- an xwidget has a plist to hold the model, like a process
+- an xwidget has an assoc list of xwidget views
+
+there are some things that arent clear:
+- an xwidget doesnt necessarily need to be coupled to a buffer but it
+ seems to be the clearest model. xwidgets would be buffer local
+- xwidget-views are by necessity coupled to a emacs window so it might
+ be better to store them window locally rather than in an assoc
+ coupled to the xwidget model
+- for some gtk widgets that resist an mvc approach, like the webkit
+ widgets, special operations are needed, similar to the old phantom
+ widgets aproach. so we need to differentiate live and phantom
+ instances for these troublesome widgets and let lisp manage all the trickery.
+
+stuff that needs to work:
+- do something for all views of a xwidget(resize, value change)
+- do something for all xw-views in an emacs window(deletion etc)
+- lookup xw-view for xwidget in emacs window(during redisplay)
+(- do something for all siblings of a xw-view. not atm)
+
+*** DONE xwidget creation interface
+ CLOSED: [2011-07-18 Mon 01:59]
+xwidgets are a little bit like emacs processes but also a little bit
+like emacs images. Therefore its not perfectly obvious how to handle
+creation. Currently I just use hardcoded identifiers. the real scheme
+needs to be something else.
+
+Heres a tentative approach:
+- xwidget-create returns a xwidget object, like process creation
+ functions. the xwidget will be largely uninitialized until
+ discovered by redisplay. an xw belongs to a buffer
+- xwidget-insert inserts the xwidget in a buffer. when discovered by
+ redisplay it will be initialized and a xwidget-view allocated
+- an event will be emitted when initialization is finished when
+ relevant like for sockets
+
+the problem with this aproach is that its not really legal to reuse
+xwidget objects by writing several display specs who reference the
+same xwidget. It could presumably be done but it would just become
+weird for no real benefit. the big preblem is that the display spec
+decides the on-screen size, and its not sane to have xwidget views
+with different sizes. therefore such display specs would need to be
+catched and dissallowed. Except it is hard because AFAIK the specs
+don't have an identity as such. A flag in the structure could be set
+by lookup so the first display attempt would win. but then you can't
+rewrite the spec to change the size. hmmm. A third approach would be
+to just allow the 1st spec refering an xw during a redisplay to take
+effect, the rest are signaled as errors. this wouldnt be too bad.
+
+the other aproach would be to work more like images:
+
+- write the display spec with all information needed to create the
+ xwidget.
+- retrieve the xwidget objet from the spec with an xwidget-at-point function. It
+ can be uninitalized which client code must handle. Unlike
+ assynchronous process creation we dont get back a handle, because
+ there is none yet.
+- emitted event on initialization, when needed. Many widgets don't
+ need this. for instance, a button sends an event when pressed. but
+ you can't press it unless its on screen, and then its initialized
+ properly.
+
+This approach seemed good, but how do I know which instance
+generates an event if I cant set the id beforehand?
+
+so, therefore, the first two aproach is used.
+
+
+*** DONE xwidget creation interface actually
+ CLOSED: [2011-07-18 Mon 01:59]
+conclusion of above ramblings:
+- should be similar to make-text-button
+- don't init from display spec, instead during make-xwidget call
+*** TODO callbacks would be nice
+but they need to be handled initially with events for technical
+reasons. C code can't call Lisp easily. The event handler can call the
+callback though.
+
+** TODO more documentation
+There should be user docs, and xwidget contributor docs. The current README
+is all contributor docs there is now, apart from the code.
+
+
+
+** CANCELLED look into more ways of displaying xwidgets, like binding them to a
+ CLOSED: [2011-07-05 Tue 11:34]
+window rather than a point in a buffer. This was suggested by Chidong.
+This would be a useful addition to Emacs in itself, and would avoid nearly all
+display issues. I still think the general case is more interesting, but this
+special case should also be added. The xwidget would then be bound to
+replace the view of a particular window, and it would only show in
+that window.
+
+I got the webkit xwidget to work well enough so I dont see the need
+for this now, except for sockets and I think it can better be dealt
+with at the lisp level.
+
+** DONE MVC mode for xwidgets
+ CLOSED: [2011-06-27 Mon 12:53]
+It appears unfruitful to chase using the same display mode for all
+types of xwidgets. Composition is fun but not robust the way I
+tried to do it.
+
+Instead there should be a set of MVC xwidgets. Each on-screen instance
+of an MVC widget would be a real GTK widget. The instances would
+communciate state using signals.
+
+There are drawbacks. There is no inbuilt support for MVC in GTK, so we
+have to roll our own, which is tedious if not much work for the few
+cases.
+
+MVC for xembedded application will need support from the applications
+themselves. Inkscape supports multiple views to the same document,
+other programs don't. In practice it might not be a big drawback.
+
+
+*** DONE figure out what to do with the multiple frames case.
+ CLOSED: [2011-06-27 Mon 12:52]
+This should be easier to solve with MVC.
+Surprisingly, this just worked!
+*** DONE how to propagate changes in views to other views?
+ CLOSED: [2011-06-27 Mon 12:53]
+I used gtk signals, the implementation for sliders works well!
+
+** TODO canvas support
+heres an interesting comparision of gtk canvases
+http://live.gnome.org/ProjectRidley/CanvasOverview
+*** goocanvas
+goocanvas is a gtk canvas implemented using cairo. investigate.
+
+pros:
+- it has a MVC model aproach out of the box which is nice.
+
+http://developer.gnome.org/goocanvas/unstable/goocanvas-model-view-canvas.html
+
+export CFLAGS="`pkg-config --cflags goocanvas` -DHAVE_GOOCANVAS"
+export LDFLAGS=`pkg-config --libs goocanvas`
+./configure
+make
+
+I made a hello goo world xwidget so seems doable.
+I wanted to load a SVG which wasnt immediately straightforward, so I
+tried clutter. but it turns out the exact same strategy could be used
+with goocanvas.
+
+*** clutter
+maybe clutter can be used as a canvas?
+pros:
+- seems to have a lot of traction atm. many examples
+- potentialy fast and cool vector graphics
+cons:
+- no out of the box MVC support, but seems doable. no worse than the
+ other home brew mvc support I have in xwidgets
+(media-explorer in an application that employes the MVC pattern)
+
+http://www.openismus.com/documents/clutter_tutorial/0.9/docs/tutorial/html/sec-stage-widget.html
+
+there is also cool stuff like this:
+http://gitorious.org/webkit-clutter/webkit-clutter which is an webkit actor for
+clutter! hmmmmm.
+
+I want to render svg. aparently:
+ librsvg rsvg_handle_render_cairo(h, cr);
+ ClutterCairoTexture
+ Clutter
+
+export CFLAGS="`pkg-config --cflags clutter-gtk-1.0` -DHAVE_CLUTTER"
+export LDFLAGS=`pkg-config --libs clutter-gtk-1.0`
+./configure
+make
+
+compiles but I get:
+Gtk-ERROR **: GTK+ 2.x symbols detected. Using GTK+ 2.x and GTK+ 3 in
+the same process is not supported
+
+export CFLAGS="`pkg-config --cflags clutter-gtk-0.10` -DHAVE_CLUTTER"
+export LDFLAGS=`pkg-config --libs clutter-gtk-0.10`
+./configure
+make
+
+
+*** webkit html 5
+expose the DOM to lisp or something. The webkit xwidget works pretty
+well now, so this might be the way ahead.
+** DONE mvc code crashes after a while
+ CLOSED: [2011-07-12 Tue 18:52]
+seemingly only when compiling with optimizations.
+I have no idea why.
+
+Doesn't seem to happen after some code cleanups.
+** DONE xwidget-resize-at
+ CLOSED: [2011-07-19 Tue 14:28]
+reimplement so display spec is not involved
+** DONE display spec validation
+ CLOSED: [2011-07-19 Tue 14:44]
+it is an error to reuse xwidgets in several buffers or in the same
+buffer. how do we catch these errors?
+- showing the same xwidget twice in a buffer is no more wrong than
+ showing in several emacs windows, just conceptually wrong, so ignore
+ this case for now
+- xwidgets now store a reference to the buffer they were created in,
+ so use that to invalidate xwidget references in oher buffers. but
+ thats not really an error either
+- xwidgets should now be proper lisp objects so you dont delete them
+ you await their garbage collection. so therefore there can never be
+ invalid disploy specs
+
+so turned out this got solved by using proper lisp objects for
+xwidgets. yay!
+
+** DONE clipping of controllers
+ CLOSED: [2011-07-05 Tue 11:33]
+
+Emacs uses a big GTK window and does its own clipping against Emacs
+windows inside this area. So, in order to layout gtk widgets in emacs
+windows we must clip thim ourselves.
+
+The following method worked well for a long time:
+- make a gtk widget, say a button, xw
+- make a clipping area, of type gtkfixed(many types have been tested)
+- put the clip area in the main emacs gtk window
+- figure out clip area changes during emacs redisplay
+
+the only weirdness was that one has to tell gtk the clip area has a
+window in order to get clipping. This is weird because all gtkwidgets
+are windows in a sense and a window is almost by definition also a
+clipping area.
+
+Anyway, in GTK3 the gtk_widget_set_has_window(GTK_WIDGET (
+xv->widgetwindow), TRUE); call is ignored.
+
+The gtkeventbox which is documented to have its own window doesnt work
+either.
+
+http://www.lanedo.com/~carlos/gtk3-doc/chap-drawing-model.html
+
+anyway clipping is rather complicated but seems to finally work okay.
+
+*** DONE subclass my own clipping widget
+ CLOSED: [2011-07-04 Mon 16:55]
+http://www.lanedo.com/~carlos/gtk3-doc/GtkWidget.html#gtk-widget-set-has-window
+mentions that it has_window can only be called inside a widget
+impementation.
+
+this wasnt really the issue. allocation was the problem
+*** DONE try scrolled window
+ CLOSED: [2011-07-01 Fri 10:56]
+clipping does in fact work with
+gtk_scrolled_window_add_with_viewport (xv->widgetwindow, xv->widget);
+!!
+
+I get unwanted scrollbars in the widget though.
+
+ gtk_scrolled_window_set_policy ( xv->widgetwindow,
+ GTK_POLICY_NEVER, GTK_POLICY_NEVER);
+
+stops clipping from working!
+
+
+*** DONE try viewport
+ CLOSED: [2011-07-01 Fri 10:56]
+gtkviewport is used in scrolled window so in order to remove
+scrollbars it should be possible to use viewport directly. however,
+viewport ignores size requests. or rather the container does.
+
+
+*** DONE debug allocation
+ CLOSED: [2011-07-04 Mon 16:56]
+the container determines how much size to allocate to child widgets.
+
+ GtkAllocation galloc;
+ gtk_widget_get_allocation(GTK_WIDGET (xv->widgetwindow), &galloc);
+ printf("allocation %d %d , %d %d\n", galloc.x,galloc.y,galloc.width,galloc.height);
+
+after my clipping attemp shows that my size request is ignored! this
+might be logical, since the container provided by emacs is a
+gtkfixed. gtkfixed might choose to heed the widgets size desires and
+allocate the entire widget size. but we want clipping!
+
+since i cant reasonably expect to change the emacs main container, i
+can maybe overide the setallocation method in gwfixed, and adjust
+allocation to clipping if its an xwidget asking for allocation.
+
+**** DONE subclass gtkfixed
+ CLOSED: [2011-07-04 Mon 16:56]
+possibly i need to subclass gtkfixed and override
-
- export CFLAGS="`pkg-config --cflags webkitgtk-3.0 ` -DHAVE_WEBKIT_OSR "
- export LDFLAGS=`pkg-config --libs webkitgtk-3.0 `
- ./configure --with-x-toolkit=gtk3
- make
-
++#+begin_src C
++ void gtk_widget_size_allocate (GtkWidget *widget,
++ GtkAllocation *allocation);
++#+end_src
+http://developer.gnome.org/gobject/stable/howto-gobject.html
+
+turns out emacs already does this for gtk3 according to jan D:
+>>For GTK3, Emacs already subclasses GtkFixed, see emacsgtkfixed.[ch].
+
+- widgets may not be underallocated, aparently
+http://mail.gnome.org/archives/commits-list/2011-April/msg10950.html
+
+- how to call base class method/chain up
+http://developer.gnome.org/gobject/stable/howto-gobject-chainup.html
+
+- the allocation modification could happen in the container or the
+ child. it feels more apropiate in the container
+
+it is however unexpectedy inconvenient to modify allocation because
+the needed data is private to the base class. to overcome this:
+
+ - run base class method 1st.
+ - then, iterate all children, and modify allocation for xwidget
+ children only. x y will then be set.
+
+JanD pointed out the GTK3 port already has its own subclass, so I
+modified that one.
+
+*** DONE clip top
+ CLOSED: [2011-07-05 Tue 11:30]
+there are four controller edges that potentialy need clipping. I begun
+with right and bottom edges. clipping them is just a matter of setting
+the right size of the widgetwindow and also ensure it gets the right
+allocation from the container.
+
+clipping top (and left) is not equally straightforward. I'm using a
+viewport now and scroll it the amount that needs to be clipped.
+however, the viewport is sensitive to changes in allocation, which
+makes it harder to use the allocation workarounds.
+
+see:
+- gtk_widget_set_size_request
+- gtkscrolledwindow
+
+I returned to using a simple gtkfixed for the widgetwindow. with
+allocation hack and set_has_window it works. Idea prefer not to have
+the allocatien hack and it wasnt needed it gtk3 only gtk2. needs
+furthi investigation,
+
+** various code cleanups
+There are many cleanups necessary before any hope of inclusion in
+Emacs trunk. To begin with, the part of the patch that touches other
+parts of emacs must be very clean.
+*** DONE use FRAME_GTK_WIDGET (f)
+ CLOSED: [2011-07-20 Wed 20:02]
+rather than gwfixed.
+
+*** DONE support configure
+ CLOSED: [2011-07-12 Tue 18:48]
+*** DONE ifdef all xwidget code
+ CLOSED: [2011-08-13 Sat 16:19]
+so you can reliably disable the code at compiletime
+** DONE translate clicks
+ CLOSED: [2011-07-03 Sun 22:12]
+on onscreen webkit peer to offscreen
+
+maybe
+http://developer.gnome.org/gdk/stable/gdk-Windows.html#GdkWindow-from-embedder
+
+turned out to be not so hard, captured events, copied them and
+forwarded them offscreen!
+
+** TODO investigate gdk_window_redirect_to_drawable
+http://developer.gnome.org/gdk/stable/gdk-Windows.html#gdk-offscreen-window-set-embedder
+maybe could be used in place of my own copy hacks? to work it must
+support a chain of redirects, which seems unlikely. the benefit would
+be that I dont have to spend time optimizing redrawing.
+
+
+** DONE remove xwidget_views when emacs window is deleted
+ CLOSED: [2011-07-05 Tue 11:29]
+removing xwidget views when an Emacs window closes is not reliable.
+
+- switching buffers in a window seems to hide the corresponding
+ xwidget-views properly, but they might as well be deleted.
+
+- patching delete-window-internal could be used to delete the xwidget-views
+this seems to work
+
+
+** browser xwidget
+although embedding a browser is not my primary concern many are
+interested in this. some suitable browser component needs to be found
+supporting gtk.
+
+*** DONE webkit
+ CLOSED: [2011-07-03 Sun 22:13]
+there is a webkit gtk port. there is no obvious mvc support.
+http://live.gnome.org/WebKitGtk
+http://webkitgtk.org/
+
+it might be possible to keep a set of webxits in artificial
+synchronisation by recursive deep copy of the DOM from one webkit to
+another. This will be error prone at best though. Another way might be
+to just use bitmap copy of the "live"instance to the "phantom"
+instances. the problem of transfering the live view remains though.
+
+export CFLAGS="`pkg-config --cflags webkit-1.0` -DHAVE_WEBKIT -g"
+export LDFLAGS=`pkg-config --libs webkit-1.0`
+./configure
+make
+
+**** off screen rendering
+export CFLAGS="`pkg-config --cflags webkit-1.0` -DHAVE_WEBKIT_OSR -g"
+export LDFLAGS=`pkg-config --libs webkit-1.0`
+./configure
+make
+
+works a little bit but i get errors like:
+
+(emacs:8362): GLib-GObject-WARNING **: invalid cast from `GdkOffscreenWindow' to `GdkDrawableImplX11'
+
+set a breakpoint in g_log, backtrace seems to indicate
+webkitViewportAttributesRecompute is the offender.
+
+maybe try gtk3 variants?
-
++#+begin_src sh
++ export CFLAGS="`pkg-config --cflags webkitgtk-3.0 ` -DHAVE_WEBKIT_OSR "
++ export LDFLAGS=`pkg-config --libs webkitgtk-3.0 `
++ ./configure --with-x-toolkit=gtk3
++ make
++#+end_src
+crash in gtk_window_get_size instead. great.
+
+http://gtkplus-p3.0.sourcearchive.com/documentation/2.91.5-0ubuntu1/testoffscreenwindow_8c-source.html
+
+after many atempts, the basic issue remains. for some reason the
+offscreen widget isnt ok when I want to snapshot it, so i simply get
+emptiness. the surface is only ok someimes.
+
+here is a useful debugging snippets:
++#+begin_src C
+ // debugging redraw:
+ // - the bg colors always change, so theres no error in signal handling
+ // - i get this error now and then:
+ //(emacs:7109): GLib-GObject-WARNING **: invalid cast from `GdkOffscreenWindow' to `GdkDrawableImplX11'
+ // seems to happen in webkit actually. see README
+
+ if(0){ //redraw debug hack. helped a lot in fact. use the with alpha painter below also
+ cairo_set_source_rgb(cr, osr_dbg_color, 1.0, 0.2);
+ cairo_rectangle(cr, 0,0, xw->width, xw->height);
+ cairo_fill(cr);
+ osr_dbg_color+=0.1;
+ if(osr_dbg_color>1.0)
+ osr_dbg_color=0.0;
+
+ }
-
++#+end_src
+you need to terminate drawing like this:
++#+begin_src C
+ //cairo_set_source_surface (cr, src_pixmap, 0,0);
+ //cairo_set_operator (cr, CAIRO_OPERATOR_OVER);
+
+ //cairo_paint_with_alpha (cr, 1.0);
+ //cairo_paint(cr);
++#+end_src
+
+the snippets change background color on oach redraw.
+
+**** on-screen rendering to separate window
+an alternative might be to open a separate window and snapshot it. the
+idea is that whatever oddness webkit does so that offscreen rendering
+doesnt work, doesnt happen on-screen. the window could be opened
+somewhere not in the way.
+
+*** CANCELLED firefox
+ CLOSED: [2011-07-03 Sun 22:13]
+http://www-archive.mozilla.org/unix/gtk-embedding.html
+seems to be severly bitrotted
+
+heres a newer aproach
+http://hg.mozilla.org/incubator/embedding/file/29ac0fe51754/gtk/tests/test.cpp
+
+while webkit clearly has the best traction as an embeddee, the
+offscreen rendering issues makes it interesting to see what ff brings
+to the table.
+
+turned out webkit has as good offscreen support as anyone, see I went
+with that in the end.
+
+
+*** DONE text field support
+ CLOSED: [2011-07-20 Wed 20:05]
+Emacs captures all keyboard events so text field support isn't super
+straightforward.
+
+**** propagate keyboard events
+I have some old hacks for this and they are not good.
+**** use the DOM model
+expose document.activeElement to lisp. This is potentially more
+interesting than just forwarding keyboard events.
+
+webkit_web_view_get_dom_document ()
+
+this is hard it seems. an idea might be to hack elisp support for swig
+to machine generate the bindings.
+**** DONE inject javascript
+ CLOSED: [2011-07-03 Sun 22:50]
+webkit_web_view_execute_script ()
+
+this works now:
+(xwidget-webkit-execute-script 5 "document.activeElement.value='test'")
+
+so it should be possible to do some interesting stuff.
+execute-script does however not return anything at the interface level
+so satisfaction is not total:
+
+http://markmail.org/message/4yowmdgras73z3x5
+
+maybe
+https://launchpad.net/gnome-seed
+
+or this funny hack:
+<jave> im trying to understanh how to interact via javascript to an embedded
+ webkit gtk instance [23:38]
+<jave> i use webkit_web_view_execute_script() which is nice but doesnt return
+ a value, by design aparently [23:39]
+<jave> any hints?
+<lucian> jave: afaik, webkit still doesn't have full gobject bindings [23:48]
+<lucian> jave: you can hack it up by making the JS modify the title, and read
+ the title from gtk-side
+<jave> lucian: that was a pretty cool idea!
+
+
+*** webkit_web_view_load_string ()
+I would like preview of html in a buffer rather than from uri.
+
+
+
+*** DONE simple xwidget-webkit wrapper
+ CLOSED: [2011-07-22 Fri 11:01]
+so that it could be used for actual browsing :)
+I dont want to reinvent too many wheels so i'd like to use existing
+emacs facilities here possible. use bindings similar to w3m(or info)
+
+- m-x xwidget-webkit starts a session
+- 'g' goes to a url
+- use bookmark-jump i suppose. I mostly use org for bookmarks myself
+- browse-url support so webkit can be the default browser
+- some way of getting around the quirky keyboard interaction since
+ xwidgets dont receive keyboard events because I hawe no idea how to
+ do that in a sane way
+
+... and one can of course go on bikeshedding forever. lets keep it
+simple and extensible, and compatible with other Emacs packages.
+
+the really cool ideas would need Emacs DOM integration, which is not
+easy.
+
+** webkit related
+*** TODO webkit support webkit signals
+
+**** DONE particularily document-load-finished
+ CLOSED: [2011-08-01 Mon 22:34]
+http://webkitgtk.org/reference/webkitgtk-webkitwebview.html#WebKitWebView-document-load-finished
+because one might need tell set a title and sizes and things when it loads.
+**** TODO event bug
+Debugger entered--Lisp error: (error "Two bases given in one event")
+
+hapens sometimes with xwidget events. appears to be when the
+originating xwidget is offscreen so that the event doesn't get caught
+by the correct emacs event map.
+
+maybe I need to set the originating window in the event structure.
+ event.frame_or_window = Qnil; //frame; //how to get the frame here? //TODO i store it in the xwidget now
+
+since its an offscreen xwidget the buffer local keymap isnt the right
+place for the handler. some global map should be used.
+
+onscreen widgets don't have the same issue.
+
+anyway, seems it'll turn out like this:
+- xwidget-osr stores a callback and user data
+- the event is an implementation detail only and get caught in the
+ topmost event map
+- the event map calls the callback in the xw with the right args.
+
+we need the event handler at some level because we can't call lisp
+asynchronously.
+
+*** TODO console messages
+http://webkitgtk.org/reference/webkitgtk-webkitwebview.html#WebKitWebView-console-message
+http://getfirebug.com/wiki/index.php/Console_API#console.count.28.5Btitle.5D.29
+because maybe we can make a simple JS REPL that way.
+ (xwidget-webkit-execute-script ( xwidget-webkit-last-session)
+ "console.log('hello')")
+prints hello to stdout but theres no way to catch stdout from webkit I
+think other than receiving the signal.
+
+*** TODO webkit flashkiller by default
+while its possible to support plugins in the webkit xwidget, flash has
+issues on 64 bit, and slows down emacs to a halt with off screen
+rendering, and of course is not free software. its in the way for real
+world usage even if its interesting to watch flash animations inside
+emacs. which should be achieved with Gnash or other free software
+instead.
+
+http://stackoverflow.com/questions/4885513/prevent-flash-in-cocoa-webview
+
+simply use this api:
+http://webkitgtk.org/reference/WebKitWebPluginDatabase.html
+
+theres an implementation now but it's not robust enough webkit often
+crashes taking emacs with it.
+
+*** TODO webkit downloads
+when clicking a download link in Webkit Emacs should take over and handle it
+from there. Probably need signals. There are Emacs libraries to
+download things, with wget etc. an url.el facility should be made.
+"download-requested"
+*** TODO webkit alt-text not handled
+XKCD use image-title to display a cartoon comment. These mysteriously
+don't work ATM. Other mouseovers work though. Maybe webkit tries to
+open a new window or something, which wont work.
+
+*** TODO webkit isearch in webkit buffers
+have a look at how docview solves it
+webkit_web_view_search_text ()
+*** TODO webkit relative references doesn't work
+probably because we handle scrolling in a non-standard way. It does
+work sort of when theres a html frameset and webkit scrolls by itself.
+
+*** TODO webkit width adjustment handling issue
+since there are so many levels of clipping and whatnot in xwidgets
+sizing issues are difficult.
+
+- an xwidget is told how large it can be by emacs. thats the end of
+ it. if the xwidget thinks otherwise it will be clipped.
+- but emacs can ask the xwidget how large it wants to be. it can then
+ resize the reserved area and inform the xwidget thusly.
+
+That should have been enough. but webkit never reports less than what
+it already has. So currently a webkit view will only growth and not
+adjust to smaller sizes.
+
+This is not a big problem in practice but is still annoying.
+
+to see the problem surface to http://www.slashdot.org
+- xwidget-webkit-adjust-size
+- xwidget-webkit-adjust-size-to-content
+
+and then compare by resizing in Epiphany, which is also webkit based.
+
+*** TODO xwidget webkit allow loading from string from emacs
+*** DONE xwidget-webkit-last-session
+ CLOSED: [2011-08-01 Mon 22:38]
+was rather hurried. end result is that the lisp layer only really
+allows for one webkit session.
+*** TODO extract DOM to lisp
+then the SHR html renderer from Gnus could render the DOM as created
+by Webkit.
+
+made a simple oxperimental DOM tree traverser. It can be expanded to
+return a lisp representation, LDOM.
+
+in order to bring lisp and DOM closer together the LDOM can include a
+mapping to the originating DOM node. so, find a node in LDOM, and the
+cell maps to the original DOM. but since the LDOM is a copy it can get
+out of sync. DOM events might help.
+*** DONE C-X b in other buffer from webkit
+ CLOSED: [2011-08-12 Fri 22:20]
+bafflingly resets the webkit view to the top. Maybe the window
+reconfiguration hook code? further mystification is added because it
+only seems to happen with ido mode enabled.
+
+in comparison with image-mode which does the right thing, I discovered
+that image-mode has special code to handle scrolling. the browser mode
+and image mode has some similarities.
+
+I made some delegation code frrom webkit mode to image mode.
+** TODO xwidget image display spec compatibility
+some history: the first version of the xwidget display spec was
+the same as an image spec. This turned out not to be fantastic because
+an xwidget is both like a process and like an image. it has a separate
+existence from display. So now the xwidget display spec is just a
+pointer to a xwidget. But then some useful functionality in Emacs
+can't be reused for xwidget, in particular image-mode.
+
+Maybe a new image type could be added that was a wraper on an
+xwidget. Then image mode could be reused for webkit mode.
+
+I tried some adaptor code in xwidget.el so webkit mode now delegates
+to image mode buh its a kludge.
+
+** socket related
+*** TODO some flickering during redisplay of sockets
+with gtk3 an size allocation workaround is used.
+this seems maybe to result in flickering sizewize y-axis with the
+xwidget socket type. The webkit xwidget doesn't seem similarily
+afflicted.
+
+the size allocation workaround works by 1st running the ordinary
+allocation then modifying the results. its done this way to minimise
+the copy paste index from the base class. it might be that the
+original allocation has a brief time window to show itself.
+
+tried to modify the allocation hack so it doesn't call allocate
+twice. this doesn't seem to help flicker at all aparently so the
+hypothesis falls. Maybe then a socket simply doesn't lke being clipped
+by gtkfixed.
+
+*** TODO xwidget view reaping too agressive
+hide an emacs window for a while and return to it. the xwidget might
+get reaped and a new socket thus created.
+*** DONE try out OSR for sockets
+ CLOSED: [2011-07-25 Mon 21:30]
+
+didn't work too well in the inkscape case. it might be that some other
+bitmap copy method works better though.
+
+basically sockets doesn't like to be offscreen because they want their
+own gdk window.
+
+** DONE synchronise emacs background with xwidget color
+ CLOSED: [2011-08-11 Thu 11:04]
+fine-tuning to reduce flicker.
+
+isn't needed if emacs bg erase disabled
+
+** DONE xwidgets doesn't work during bootstrap all of a sudden
+ CLOSED: [2011-08-01 Mon 22:33]
+might be some annoying local issues with my install because it is not
+reliably reproducible. (went away during merges)
+
+** TODO low impact xwidget based image viewer
+for instance to render SVG using webkit, or some other canvas.
+that way it would be possible to merge to trunk in stages.
+
+so, webkit could be used to display the SVG. the display spec for
+images would be used. multiple webkits would be used rather than
+offscreen rendering, so it would be GTK2 compatible.
+** DONE xwidget movement doesn't work all of a sudden
+ CLOSED: [2011-08-11 Thu 11:03]
+this used to work great. now it doesn't.
+
+suspects:
+- XCopyArea
+ - x_shift_glyphs_for_insert
+ - x_scroll_run. this is run by the try_window* functions, and
+ inhibiting them dösnt help. but also callid in scrolling_window.
+
+
+- try_window_reusing_current_matrix
+- I used to enable GLYPH_DEBUG which I currently don't. it disables
+ many optimisations. this was fixed.
+- lookup_xwidget then produce_xwidget_glyph gets called always but not
+x_draw_xwidget_glyph_string probably because of scroll optimization.
+movement detection could possibly be moved to produce_xwidget_glyph(not)
+
+no longer helps:
+(setq inhibit-try-window-id t)
+(setq inhibit-try-window-reusing t)
+
+workaround:
+(run-with-timer 1 1 'redraw-display)
+
+seems to work:
+inhibiting scrolling_window(). and this seem to be enaugh to restore
+old behaviour, GLYPH_DEBUG doesn't seem needed.
+
+
+** DONE GLYPH_DEBUG doesn't work
+ CLOSED: [2011-08-08 Mon 17:30]
+was stupid accidental line removal that was hard to spot
+** TODO osc xwidget example
+a couple of xwidget sliders that control a csound/supercollider song with osc.
+so, for that to work we need slider callbacks to work. when a slider
+changes send an osc message. use ocssend:
+
+ oscsend localhost 7777 /sample/address iTfs 1 3.14 hello
+
+or better:
+http://delysid.org/emacs/osc.el
+
+sliders could be defined in csound comments or something to illustrate
+the point. or if real fanciness is desired parse the csound source
+with Semantic and provide a control buffer corresponding to the
+defined controls.
+
+
+
+Added: [2011-08-11 Thu 10:53]
+
+
+** TODO SEB
+the SEB site does something funny so I can't insert text in fields
+