Announcement

Do not use the forums to submit bug reports, feature requests or patches, submit a New Ticket instead.

#1 2005-10-15 13:13:39

tteddo
Xinha Community Member
From: Maine
Registered: 2005-06-28
Posts: 13
Website

Apostrophes

I have a strange problem with apostrophes in text that has been through Xinha. If you check the original page here:
http://www.potbellyjunction.com/html/stoves_pellet.html you see the apostrophies for "it's" and "doesn’t" appear normal in the first paragraph.
The new page on the new site, after copying and pasting just like by client is going to, appears here:
http://www.potbellyjunction.com/PelletStoves.cfm. See how there is an invalid character instead of the apostrophes? It shows like a box in Internet Exploder, nothing in Firefox-Windows, and as an invalid character in Firefox for Linux.
The database (Access 2000) field it is going into is set for Memo, so that leaves it untouched, and if I paste the text into Notepad and manually change them to " ' " ahead of time it is fine.
Thanks for your help!

Offline

#2 2005-10-22 12:04:16

tteddo
Xinha Community Member
From: Maine
Registered: 2005-06-28
Posts: 13
Website

Re: Apostrophes

Ok, maybe I should rephrase this. What are those characters (ASCII wise?) so I can replace them with the proper ones before putting the text in the database?

Offline

#3 2005-10-25 23:52:03

gogo
Xinha Leader
From: New Zealand
Registered: 2005-02-11
Posts: 1,015
Website

Re: Apostrophes

You have a character set mismatch I expect.  The second URL you give is served as UTF-8, that's good.  Check that the page you have Xinha running on is also served as UTF-8.


James Sleeman

Offline

#4 2005-11-12 17:42:10

tteddo
Xinha Community Member
From: Maine
Registered: 2005-06-28
Posts: 13
Website

Re: Apostrophes

Sorry for the delay! Got busy there for awhile!
I added <meta http-equiv="Content-Type" content="text/html; charset=UTF-8"> to the header in the control panel and it is stll the same. I also made sure it wasn't the Access database by posting directly to the next page.
I also tried changing the character set to <meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1"> like the original site that shows them properly, and that had the same effect.
Could those characters be from a different character set?

Offline

#5 2005-11-13 20:31:40

gogo
Xinha Leader
From: New Zealand
Registered: 2005-02-11
Posts: 1,015
Website

Re: Apostrophes

There is that windows character set that gets used quite often... ummm....Windows-1252, could be that?


James Sleeman

Offline

#6 2005-11-15 10:53:17

tteddo
Xinha Community Member
From: Maine
Registered: 2005-06-28
Posts: 13
Website

Re: Apostrophes

I tried that, and ISO-8859-1 (like this site) and ISO-8859-15 to the same end.
Here's something interesting. If I change it to ISO-8859-1, then post directly to another page it is fine (it wasn't before). But, if it goes into the database it isn't. I turned off Unicode Compression for that field also.
So, I guess now it must be a database problem with Access.

Offline

#7 2005-11-16 05:51:07

ray
Xinha Administrator
From: Germany
Registered: 2005-03-23
Posts: 521
Website

Re: Apostrophes

I suggest you to use valid HTML to prevent such problems.
This is from the W3C Validator (http://validator.w3.org/check?verbose=1 … ellet.html):

Error  Line 91 column 245: non SGML character number 146.

...nce you experience a pellet stove, its easy to understand why people rave a

You have used an illegal character in your text. HTML uses the standard UNICODE Consortium character repertoire, and it leaves undefined (among others) 65 character codes (0 to 31 inclusive and 127 to 159 inclusive) that are sometimes used for typographical quote marks and similar in proprietary character sets. The validator has found one of these undefined characters in your document. The character may appear on your browser as a curly quote, or a trademark symbol, or some other fancy glyph; on a different computer, however, it will likely appear as a completely different character, or nothing at all.

Your best bet is to replace the character with the nearest equivalent ASCII character, or to use an appropriate character entity. For more information on Character Encoding on the web, see Alan Flavell's excellent HTML Character Set Issues reference.

This error can also be triggered by formatting characters embedded in documents by some word processors. If you use a word processor to edit your HTML documents, be sure to use the "Save as ASCII" or similar command to save the document without formatting information.

Offline

#8 2005-11-16 10:27:29

tteddo
Xinha Community Member
From: Maine
Registered: 2005-06-28
Posts: 13
Website

Re: Apostrophes

Thanks for the info!
The problem is users in my content management system copy and paste from all sorts of things, including Word, WordPerfect, web sites, etc. and I have no control over them putting in non-standard characters.

If I had a way to identify the bad characters I could replace them with the proper quote characters before insertion into the database. I have done this before with characters that didn't sit well in a text field.
I am pretty sure they are some sort of Microsoft thing because I have the same problem on a Town website here that they copy/paste out of Word all the time.

The various "cleaning" functions in Xinha don't clean that up either, and I really wouldn't expect it to.

Offline

#9 2007-05-20 13:59:18

tteddo
Xinha Community Member
From: Maine
Registered: 2005-06-28
Posts: 13
Website

Re: Apostrophes

I know it's been awhile, but I found the answer to this. Basically Access and MySQL default to Latin 1 code page and that doesn't include the smart quotes from Word. If you make the tables default to UTF8 then you should be all set. This article helped me alot: http://www.joelonsoftware.com/articles/Unicode.html

Last edited by tteddo (2007-05-20 13:59:50)

Offline

#10 2007-05-21 10:51:24

ray
Xinha Administrator
From: Germany
Registered: 2005-03-23
Posts: 521
Website

Re: Apostrophes

Unless you want to change all your content to UTF-8, you might want to try the HtmlEntities plugin

Offline

#11 2007-05-21 11:15:39

tteddo
Xinha Community Member
From: Maine
Registered: 2005-06-28
Posts: 13
Website

Re: Apostrophes

That worked great Ray!
Thanks alot!

Offline

Board footer

Powered by FluxBB