Serious Hacking and Slashing - Unified Backend Branch

Yermo · 2005-04-16 02:57:39

Gogo,

I've committed my first round of changes to the Unified Backend branch. Mostly prep work while I come up to speed on the beast.

Being a relative newbie to serious javascript development and especially Xinha internals, I found the code completely unapproachable. From a cold start, it's quite a daunting challenge to figure out how everything fits together and in what order things happen.

htmlarea.js was and is a serious mess. I figure since it's my own experimental branch I can probably get away with hacking it to death without annoying too many people.

I find phpdoc very useful and wanted a JavaScript analogue. I found a tool called JSDoc that does a reasonable job:

http://jsdoc.sourceforge.net

So now all methods in htmlarea.js have the beginnings of JavaDoc style headers. To see what the output looks like check out:

http://www.formvista.com/uploaded_html/ … index.html

It's not perfect but it's the best I've been able to find so far. I preserved all the comments that were in the code and where I understood what was going on I added a few of my own.

(FYI: The phpdoc documentation is currently broken).

I've also ported over a debugging trace message class I call _ddt() that has served me extremely well in other large projects. This class allows you to incorporate debugging trace messages into the code and turn the message on and off on an object by object basis. So far I've only added messages to htmlarea.js; but as I work with each plugin I'll be incorporating more trace messages. I am envisioning a debug panel where you can turn debug messages on and off for each plugin but that may be getting a little too fancy.

One DDT object is used for static methods and another DDT object is built for each HTMLArea instance. In the first case trace messages go into a popup window so you can see the entire construction process as it's happening. In the second case text areas are created for the per-object trace messages.

Messages are turned on and off via called to _ddtOn() and _ddtOff(). For convenience I've bound both types of debug trace messages to additional <> buttons on the toolbar. For an example of how this works check out:

http://www.formvista.com/uploaded_html/ … index.html

and click on the simple examples link. (I have the full_example turned off.) I didn't have extra icons lying around so for the moment I just have the <> placeholders. Hover over them to see which is which. You can click on them to bring up the debug window and type into an editor. It's dirt slow but does give an idea of what's going on; invaluable if you're stuck trying to figure out why something is breaking.

I haven't tested in IE, only firefox.

You can also turn on debug messages in the new examples/simple_example.html file if you want to see how things are constructed.

I get the impression some methods are being called far more often than they need to be.

On the todo list is to create a version of a "buildruntime.php" script that will generate a runtime stripped version of the code without comments and trace messages. The general concept is to do development on the trace enabled one and then produce a runtime when one is ready to release a build.

So that I can see what version of the software I'm dealing with I've added an "svn_commit.pl" script that acts as a front end to "svn commit" and increments a counter in popups/about.html. This causes about.html to be out of date for each commit so it will always contain the current rev number, instead of being stuck at feb 19th.

I've also added a makedocs.pl script to generate the documentation. You need perl, php, phpdoc and JSDoc in order for it to work; it's a start at least.

So that users don't have to edit the paths in these scripts I've included a Configure.php script that generates all that for them. I'm also using it to fix up some permissions issues (ImageManager/demo_images). I will probably expand Configure.php quite a bit for setting up backend support.

If you have a chance to look it over, I'd be interested in what you think about the approach. (caveat, the editor I use, CRISP, is blazingly fast but flakes a bit on tab to space conversions so you may see the occassional bracket out of place. I'll fix those as I find them.)

Finally, my coding style and approach is a bit different than most and I understand comletely that all this work may not be adopted into the mainline codebase.

-- Yermo

guillaumed · 2005-04-16 16:31:07

I think also that Yermo is right.. The code needs some cleaning and documentation.. And I've seen too many lines where the comment is or should have been : I've done it but I don't know why.. It may been done before but why not twice, ...

Imo this means that main guidelines for coding and versionnning (another thread) should be define soon...

willeffects · 2005-04-16 18:03:46

Great work Yermo. I dont think anyone will mind at all that you are fixing up the mess of htmlarea. It appeared to me xinha was just adopted and had not been rewritten for performance or ease of use since htmlarea, so this is definitely needed given the amount of development that needs to be done to get xinha where we all want to see it.

Have you already seen a performance increase in your version?

Will

Yermo · 2005-04-16 18:18:34

willeffects wrote:

Great work Yermo. I dont think anyone will mind at all that you are fixing up the mess of htmlarea.

Thanks.

willeffects wrote:

It appeared to me xinha was just adopted and had not been rewritten for performance or ease of use since htmlarea.

Definitely true. There's a tremendous amount of work to do; after about 9 months of hell work I've got a moments break so I'm working on Xinha fulltime at the moment.

willeffects wrote:

Have you already seen a performance increase in your version?
Will

My focus right now is just making the code easier to work with before I start mucking with changing the client server communications.

However, it is pretty amazing how the performance degrades proportional to the number of plugins that are loaded ... but I think the trace message infrastructure will help alot in determining where optimizations can be made.

I figure it'll be at least another two weeks before I'm "finished" with the first Unified Backend release. In addition to routing everything through a single backend script I'm going to be fixing alot of bugs and improving EnterParagraphs.

PeteBest · 2005-04-18 09:54:05

Sounds like you've got a lot of hard work ahead of yourself there Yermo! Hopefully the changes from the stripped down version of this will make it's way into the stable release. Imagine that'll help out a lot with seperating Xinha into browser specific files. Good work!

willeffects · 2005-04-18 15:54:42

Yeah I would definitely suggest to look at how to break things out in browser specific files now. With Safari 2 coming I know there are going to be plenty of users on it and that alone is going to cause a support headache for all of us.

Will

niko · 2005-04-19 01:58:53

good work Yermo! nice to see getting xinha more and more improved
(code- and performace optimisation is surely much difficultier than writing new features...)

when will safari 2 come? will it be included in Tiger?

willeffects · 2005-04-21 21:29:11

I believe safari 2 will be in Tiger. I am not positive though.

Yermo · 2005-04-21 22:15:43

willeffects wrote:

Yeah I would definitely suggest to look at how to break things out in browser specific files now. With Safari 2 coming I know there are going to be plenty of users on it and that alone is going to cause a support headache for all of us.
Will

The codebase seems in pretty bad shape to me. I'm still working through it trying to come up to speed. As I see it, before we undertake a large re-engineering project there are a few things that have to be accomplished:

1. set the code up so that it's maintainable. My approach is to comment everything I can, establish a trace
message architecture and standardize client/server communications. Others may have a different approach
that they would like to see.

2. create a stable release including a core set of plugins that works with MSIE and Gecko as we want it to with
all major bugs and usability problems solved. Top of my list is EnterParagraphs and relative vs. absolute
vs. fully qualified link handling.

3. create documentation to cover the plugin API and general code architecture to make it easier for newbies
like myself to come up to speed.

Once these things are done and we have the codebase under control we'll be in a much better position to break out the browser specific code. From my relatively inexperienced point of view, it seems that doing this might be tricky since there isn't a one-to-one mapping of functionality between all browsers.

In the meantime, I'm making slow progress on the unified backend branch. I'll probably commit another round of changes to it some time next week. At least for me, having trace message everywhere is making it much easier to see how the code works ... unfortunately it does make things painfully slow when all messages are turned on.

niko · 2005-04-22 02:09:11

hi yermo,

please keep in mind that one day we should merge the trunk and your branch!
how will we do that?

did you have a look at http://xinha.gogo.co.nz/punbb/viewtopic.php?id=163 ?

and what id your opinion about that?
- where should HTMLArea.InsertImageDialog be defined?
- should the dialog-html-code loaded on generate or when clicking the button?

guillaumed · 2005-04-22 03:35:37

Hi to all,

Seems that we all should consider that there is a lot of thing to do but also that we want to do it... Great..

My three cents :

1. The concept of plugin is very important to have a dedicated or evolving editor, but that's mean that there is a clear interface between the core editor and the plugin (as I have seen there a specific behavior in the editor core and some plugins..)

2. Efficiency : we have to think of the burden for the browser of increasing the functionality : we should avoid to load all the plugins at start, developping client/server interaction when not necessary, ...). We should define some partical guidelines for that. To my point of view Niko, you should load the code only when necessary so when clicking the button...

But we are at a normal stage of an open source project...!!

niko · 2005-04-22 05:12:50

To my point of view Niko, you should load the code only when necessary so when clicking the button...

gogo's oppinion is to load it on generate (if i can speak for him here)
- the editor responds much faster when everything is preloaded
- the loading of the additional html-files will be done in the background - so the user can work allready with xinha.

guillaumed · 2005-04-22 08:16:05

niko wrote:

the loading of the additional html-files will be done in the background

How can you do that?? Using the timeout method doesn't fullfill that..

niko wrote:

the editor responds much faster when everything is preloaded

see previous Yermo remark :

Yermo wrote:

However, it is pretty amazing how the performance degrades proportional to the number of plugins that are loaded

mokhet · 2005-04-22 09:12:52

guillaumed wrote:

1. The concept of plugin is very important to have a dedicated or evolving editor

Yes, the concept of plugin is required. We all need an editor but none of us need the same behavior and features. Plugins are answering to this concern.

niko wrote:

To my point of view Niko, you should load the code only when necessary so when clicking the button...
gogo's oppinion is to load it on generate (if i can speak for him here)

of course it's going smoother when everything is preloaded, but the core is already 146Ko, it's a lot of javascript to transfer. Loading also the plugins logic is too much i think.

Of course this "argue" of the actual size of core will not be valid when the discussion about the separation by navigators will be complete, but that's bring me to an interrogation. Perhaps the structure of plugins could be separate in two files. One init file which could be preloaded by core, and the other one with the remaining code that would be loaded on demand only. Well, as always it's pure js speculation ehe

guillaumed · 2005-04-22 10:11:17

mokhet wrote:

One init file which could be preloaded by core, and the other one with the remaining code that would be loaded on demand only.

I totally agree on that...

Yermo · 2005-04-23 00:30:27

niko wrote:

hi yermo,
please keep in mind that one day we should merge the trunk and your branch!
how will we do that?

I'm beginning to wonder that myself.

niko wrote:

did you have a look at http://xinha.gogo.co.nz/punbb/viewtopic.php?id=163 ?
and what id your opinion about that?
- where should HTMLArea.InsertImageDialog be defined?
- should the dialog-html-code loaded on generate or when clicking the button?

I haven't gotten that far yet. I'm still getting a handle on how everything fits together.

I have, however, made some radical improvements to the trace system which now means my branch is useable even with all the trace messages turned on. (The version in SVN right now is ridiculously slow).

In addition, I've added a preprocessor that patches in file and line numbers to all debugging trace messages. It's making it wicked easy to see how everything works ... interestingly the full_example.html has a bug, it's generating everything twice since xinha_init is being called on body.onLoad and once on window.onLoad.

Actually, from the trace messages one can immediately see where Xinha is doing a tremendous amount of repetitive processing.

I've got some more hacking to do and then I'll do another commit. For anyone trying to come up to speed on how Xinha works the next commit of the unified backend branch might be a much better place to start.

For a taste of what a partial startup trace of a single Xinha editor with the Stylist and ContentMenu plugins looks like (I haven't changed the order of how anything is called ... but the line numbers in my branch don't match the trunk at all since I've moved everything around) - something to note, see how loadPlugins() keeps cloning the plugins array as it recurses? .... :

(startup) htmlarea.js:828 - loadScript(): Top with url 'http://formvista.yml.com/TESTS/xinha_ub/dialog.js' and plugin 'undefined'
(startup) htmlarea.js:828 - loadScript(): Top with url 'http://formvista.yml.com/TESTS/xinha_ub/inline-dialog.js' and plugin 'undefined'
(startup) htmlarea.js:828 - loadScript(): Top with url 'http://formvista.yml.com/TESTS/xinha_ub/popupwin.js' and plugin 'undefined'
(startup) htmlarea.js:1263 - loadStyle(): top with style 'htmlarea.css' and plugin 'undefined'
(startup) htmlarea.js:1281 - loadStyle(): appending 'http://formvista.yml.com/TESTS/xinha_ub/htmlarea.css' to document
(startup) htmlarea.js:872 - init(): top
(startup) htmlarea.js:937 - init(): calling first loadNextScript()
(startup) htmlarea.js:916 - loadNextScript(): loading 'http://formvista.yml.com/TESTS/xinha_ub/dialog.js'
(startup) htmlarea.js:916 - loadNextScript(): loading 'http://formvista.yml.com/TESTS/xinha_ub/inline-dialog.js'
(startup) simple_example.html:71 - Setting up xinha_init()
(startup) htmlarea.js:916 - loadNextScript(): loading 'http://formvista.yml.com/TESTS/xinha_ub/popupwin.js'
(startup) htmlarea.js:927 - loadNextScript(): end of list reached. Firing HTMLAreaonLoad handler which is 'undefined'
(startup) simple_example.html:76 - xinha_init called from window.onload handler for simple_example.html
(startup) simple_example.html:92 - calling HTMLArea.loadplugins()
(startup) htmlarea.js:1160 - loadPlugins(): top - cloning plugins array.
(startup) htmlarea.js:1342 - cloneObject(): top
(startup) htmlarea.js:1353 - cloneObject(): contructing an array object.
(startup) htmlarea.js:1369 - cloneObject(): copying object members.
(startup) htmlarea.js:1101 - loadPlugin(): loading plugin 'Stylist'
(startup) htmlarea.js:1132 - loadPlugin(): callback defined. Using _loadback() to load plugin
(startup) htmlarea.js:1790 - _loadback(): top with src 'http://formvista.yml.com/TESTS/xinha_ub/plugins/Stylist/stylist.js'
(startup) htmlarea.js:1812 - _loadback(): script tag to load javascript file appended to head section.
(startup) htmlarea.js:1221 - loadPlugins(): end
(startup) htmlarea.js:1160 - loadPlugins(): top - cloning plugins array.
(startup) htmlarea.js:1342 - cloneObject(): top
(startup) htmlarea.js:1353 - cloneObject(): contructing an array object.
(startup) htmlarea.js:1369 - cloneObject(): copying object members.
(startup) htmlarea.js:1101 - loadPlugin(): loading plugin 'ContextMenu'
(startup) htmlarea.js:1132 - loadPlugin(): callback defined. Using _loadback() to load plugin
(startup) htmlarea.js:1790 - _loadback(): top with src 'http://formvista.yml.com/TESTS/xinha_ub/plugins/ContextMenu/context-menu.js'
(startup) htmlarea.js:1812 - _loadback(): script tag to load javascript file appended to head section.
(startup) htmlarea.js:1221 - loadPlugins(): end
(startup) htmlarea.js:1263 - loadStyle(): top with style 'menu.css' and plugin 'ContextMenu'
(startup) htmlarea.js:1281 - loadStyle(): appending 'http://formvista.yml.com/TESTS/xinha_ub/plugins/ContextMenu/menu.css' to document
(startup) htmlarea.js:1160 - loadPlugins(): top - cloning plugins array.
(startup) htmlarea.js:1342 - cloneObject(): top
(startup) htmlarea.js:1353 - cloneObject(): contructing an array object.
(startup) htmlarea.js:1369 - cloneObject(): copying object members.
(startup) htmlarea.js:1192 - loadPlugins(): no plugins left to load
(startup) simple_example.html:76 - xinha_init called from window.onload handler for simple_example.html
(startup) simple_example.html:92 - calling HTMLArea.loadplugins()
(startup) htmlarea.js:1160 - loadPlugins(): top - cloning plugins array.
(startup) htmlarea.js:1342 - cloneObject(): top
(startup) htmlarea.js:1353 - cloneObject(): contructing an array object.
(startup) htmlarea.js:1369 - cloneObject(): copying object members.
(startup) htmlarea.js:1177 - loadPlugins(): plugin 'Stylist' was already loaded
(startup) htmlarea.js:1177 - loadPlugins(): plugin 'ContextMenu' was already loaded
(startup) htmlarea.js:1192 - loadPlugins(): no plugins left to load

Yermo · 2005-04-26 16:23:55

I could have sworn I posted a status update on the forum yesterday, but I may have spaced it.

Anyways, I've committed a bunch of changes to the unified_backend branch. Trace messages are much faster now such that you can use the editor with all trace messages turned on. They now all go to a single popup window. I've added a preprocessing script that patches in file and line numbers into each debugging message so you can see where each message was generated.

I've put up a small demo. Make sure to turn off any popup blockers:

http://www.formvista/uploaded_html/xinh … index.html

Click on the Simple Example. To get an idea of what Xinha does for any given action, you can click the "add separator" link in the trace window and then go do something in Xinha and check the trace window again. It's a nice way to see how event propagate through the code.

I've found it extremely useful in coming up to speed on how all this works.

If you want to look at the code and you have a subversion client you can download it using:

svn co http://svn.xinha.python-hosting.com/bra … ed_backend

If you have write access I would ask that you not post patches directly to the branch. Send them to me instead using:

http://www.formvista.com/contact.html

This branch is now severely out of date with respect to the trunk. Once I finish the work I've got on my plate I'll see about merging in all those changes to bring it up to date.

On my todo list:

- the buildruntime.php script to strip out all the comments and debugging messages to create a "runtime" version of the code.

- actually do the central backend.php script that all plugins will route through (and create placeholders for other backend languages to be used). Initially I'll do Linker and SpellChecker.

- fix up the linker plugin to route all URLs through a central processor so we can avoid having fully qualified links in the content.

- fix EnterParagraphs (big job).

If you have any questions or comments please post them here or contact me using the form above. If you look at this branch I'd be curious to know what you think. My coding style tends to be really simple and verbose .. which may not be to everyone's taste.

thanks,

-- Yermo

Xinha Discussion Forum

Announcement

#1 2005-04-16 02:57:39

Serious Hacking and Slashing - Unified Backend Branch

#2 2005-04-16 16:31:07

Re: Serious Hacking and Slashing - Unified Backend Branch

#3 2005-04-16 18:03:46

Re: Serious Hacking and Slashing - Unified Backend Branch

#4 2005-04-16 18:18:34

Re: Serious Hacking and Slashing - Unified Backend Branch

#5 2005-04-18 09:54:05

Re: Serious Hacking and Slashing - Unified Backend Branch

#6 2005-04-18 15:54:42

Re: Serious Hacking and Slashing - Unified Backend Branch

#7 2005-04-19 01:58:53

Re: Serious Hacking and Slashing - Unified Backend Branch

#8 2005-04-21 21:29:11

Re: Serious Hacking and Slashing - Unified Backend Branch

#9 2005-04-21 22:15:43

Re: Serious Hacking and Slashing - Unified Backend Branch

#10 2005-04-22 02:09:11

Re: Serious Hacking and Slashing - Unified Backend Branch

#11 2005-04-22 03:35:37

Re: Serious Hacking and Slashing - Unified Backend Branch

#12 2005-04-22 05:12:50

Re: Serious Hacking and Slashing - Unified Backend Branch

#13 2005-04-22 08:16:05

Re: Serious Hacking and Slashing - Unified Backend Branch

#14 2005-04-22 09:12:52

Re: Serious Hacking and Slashing - Unified Backend Branch

#15 2005-04-22 10:11:17

Re: Serious Hacking and Slashing - Unified Backend Branch

#16 2005-04-23 00:30:27

Re: Serious Hacking and Slashing - Unified Backend Branch

#17 2005-04-26 16:23:55

Re: Serious Hacking and Slashing - Unified Backend Branch

Board footer