33 Commits

Author SHA1 Message Date
Ruben Vermeersch
fbe773c636 Release v1.0.1 2015-09-21 09:35:02 +02:00
Ruben Vermeersch
7ceda82794 Merge pull request #17 from rubenv/fix_issue16
fix issue #16
2015-09-21 09:34:41 +02:00
Julian Bäume
d1be0f51b0 fix issue #16
add a blank line in the comments of the headers of the big.po file. This
breaks parsing the headers. msgcat of gettext tools just ignores empty
lines until a header is found and treats everything above as header
comment.

The change to the library itself fixes the tests again, after the blank
linke broke 3.
2015-09-21 09:13:43 +02:00
Ruben Vermeersch
869f763d80 Release v1.0.0 2015-08-20 15:13:21 +02:00
Ruben Vermeersch
b9394176b1 Handle extracted comments in headers 2015-08-20 15:12:56 +02:00
Ruben Vermeersch
9060221403 Release v0.3.0 2015-07-17 10:36:14 +02:00
Ruben Vermeersch
63c4209cd5 Clean up test names 2015-07-17 10:35:56 +02:00
Ruben Vermeersch
104d114d5d Only write flags when they evaluate to true. 2015-07-17 10:35:25 +02:00
Ruben Vermeersch
db5e540824 Merge pull request #14 from rubenv/fix_issue13
add tests to better document reference comments
2014-08-26 10:38:15 +02:00
Julian Bäume
cfc9b2ae82 add tests to better document reference comments
pofile does not process reference comments in any way, since the format of
references is not exactly specified. This test specifies, what users of
pofile can expect the library to do.
2014-08-12 17:47:47 +02:00
Ruben Vermeersch
2e1640d847 Release v0.2.12 2014-07-18 10:55:53 +02:00
Ruben Vermeersch
b499b7f449 Rebuild for #12. 2014-07-18 10:55:15 +02:00
Ruben Vermeersch
e42dc28fd2 Merge pull request #12 from Open-Xchange-Frontend/cstring_escapes
Cstring escapes
2014-07-18 10:54:52 +02:00
Julian Bäume
e1742e66a6 properly escape all unprintable characters
writing messages should no be in line with gettext tools. I tested
using msgcat, it provides the same results.

For some common use-cases I wrote explicit tests, for uncommon and
even unwanted use-cases I wrote one test to make sure pofile works
like msgcat for those messages
2014-06-23 18:24:02 +02:00
Candid Dauth
4cfebdee80 Fixed unescaping of all escaped C String characters.
Signed-off-by: Julian Bäume <julian@svg4all.de>

Conflicts:
	lib/po.js
2014-06-23 15:03:58 +02:00
Julian Bäume
d8fc514359 don't remove \n characters from written po file
in Item.toString, all \n characters are removed from the output.
The gettext tools however leave those characters intact. This
will now produce the same output as tools like msgcat.
2014-06-23 15:03:58 +02:00
Ruben Vermeersch
d202e39a60 Release v0.2.11 2014-06-23 14:12:25 +02:00
Ruben Vermeersch
f5056bc57f Merge pull request #11 from Open-Xchange-Frontend/fix_newline_in_msgid
restore previous behaviour with \n in strings
2014-06-23 14:11:52 +02:00
Julian Bäume
af94d8ff5e restore previous behaviour with \n in strings
An incompatible change (that actually breaks po parsing after writing) had
been introduced with commit e164fcfe9d. If
_process returned an array (which is the case for strings containing \n
character), array.toString will return a comma separated list, which is not
valid po syntax. Added a test to restore the behaviour from before the
e164fcfe9d.
2014-06-23 12:30:04 +02:00
Ruben Vermeersch
8e49417916 Release v0.2.10 2014-06-19 17:18:55 +02:00
Ruben Vermeersch
b9534d20fe Recompile. 2014-06-19 17:18:45 +02:00
Ruben Vermeersch
1b5668b5c7 Merge pull request #10 from Open-Xchange-Frontend/handle_empty_comments
handle empty comments correctly
2014-06-19 17:18:01 +02:00
Julian Bäume
08e7db58b3 handle empty comments correctly
since the lines in the parser have all newline characters removed, \s+ will
not match empty comments.

Added an example that makes other tests fail without this patch.
2014-06-19 17:04:03 +02:00
Ruben Vermeersch
eeb1382dfb Release v0.2.9 2014-06-19 14:16:52 +02:00
Ruben Vermeersch
656bfd0b8d Add JSCS, fix style. 2014-06-19 14:16:30 +02:00
Ruben Vermeersch
03572f9711 Merge pull request #9 from Open-Xchange-Frontend/fix_commented_obsolete_strings
Fix commented obsolete strings
2014-06-19 14:05:32 +02:00
Ruben Vermeersch
6af1cf741f Remove node 0.8 from Travis. 2014-06-19 14:02:54 +02:00
Julian Bäume
e164fcfe9d new implementation of items marked obsolete
the current implementation of items marked obsolete did not allow "plain"
comments for these items. However, this is perfectly fine according to
the original gettext tools. When writing a po file, comments for obsolete
items don't contain the '#~ ' mark (tested using msgcat), so this is now
also aligned with the behaviour of the original gettext tools.

For all these cases I added examples in the po files, that failed with the
current implementation and work fine after these changes.
2014-06-19 13:47:30 +02:00
Julian Bäume
8d40e1b3c4 unify string quotes 2014-06-17 21:19:14 +02:00
Ruben Vermeersch
04fe915389 Release v0.2.8 2014-03-21 11:08:59 +01:00
Ruben Vermeersch
2fb07fb1d0 Recompile browser versions for #8. 2014-03-21 11:08:30 +01:00
Ruben Vermeersch
06571c89fc Merge pull request #8 from Open-Xchange-Frontend/multiline_headers
add support for multiline string in headers
2014-03-21 11:08:05 +01:00
Julian Bäume
73b267b3e8 add support for multiline string in headers
Some languages (such as Polish, Russian or Romanian) do have more
complicated plural forms. Those are still expressible by a more
complicated mathematical expression. However, the msgmerge tool of
gettext will in these cases write multiline header fields. When parsing
such files with this lib, the headers get screwed up, so this patch
provides an example (from a pl_PL po file) and fixes this by joining the
lines in the header, before doing the actual parsing.
2014-03-21 10:54:32 +01:00
16 changed files with 567 additions and 203 deletions

34
.jscs.json Normal file
View File

@@ -0,0 +1,34 @@
{
"requireCurlyBraces": ["if", "else", "for", "while", "do", "try", "catch"],
"requireSpaceAfterKeywords": ["if", "else", "for", "while", "do", "switch", "return", "try", "catch"],
"requireParenthesesAroundIIFE": true,
"requireSpacesInFunctionExpression": {
"beforeOpeningCurlyBrace": true
},
"requireSpacesInAnonymousFunctionExpression": {
"beforeOpeningRoundBrace": true
},
"disallowSpacesInNamedFunctionExpression": {
"beforeOpeningRoundBrace": true
},
"disallowSpacesInFunctionDeclaration": {
"beforeOpeningRoundBrace": true
},
"disallowMultipleVarDecl": true,
"requireSpacesInsideObjectBrackets": "all",
"disallowSpaceAfterObjectKeys": true,
"requireCommaBeforeLineBreak": true,
"disallowSpaceBeforeBinaryOperators": [","],
"disallowSpaceAfterPrefixUnaryOperators": ["++", "--", "+", "-", "~", "!"],
"disallowSpaceBeforePostfixUnaryOperators": ["++", "--"],
"requireSpaceBeforeBinaryOperators": ["+", "-", "/", "*", "=", "==", "===", "!=", "!==", ">", ">=", "<", "<="],
"requireSpaceAfterBinaryOperators": [",", "+", "-", "/", "*", "=", "==", "===", "!=", "!==", ">", ">=", "<", "<="],
"validateQuoteMarks": true,
"validateIndentation": 4,
"disallowTrailingWhitespace": true,
"disallowKeywordsOnNewLine": ["else"],
"requireCapitalizedConstructors": true,
"safeContextKeyword": "self",
"requireDotNotation": true,
"disallowYodaConditions": true
}

View File

@@ -1,6 +1,5 @@
language: node_js
node_js:
- "0.8"
- "0.10"
- "0.11"
before_install:

View File

@@ -5,6 +5,7 @@ module.exports = (grunt) ->
@loadNpmTasks('grunt-contrib-jshint')
@loadNpmTasks('grunt-contrib-uglify')
@loadNpmTasks('grunt-contrib-watch')
@loadNpmTasks('grunt-jscs-checker')
@loadNpmTasks('grunt-mocha-cli')
@initConfig
@@ -16,6 +17,13 @@ module.exports = (grunt) ->
options:
jshintrc: '.jshintrc'
jscs:
src:
options:
config: '.jscs.json'
files:
src: [ 'lib/*.js', 'test/*.js' ]
watch:
all:
options:
@@ -49,5 +57,5 @@ module.exports = (grunt) ->
pushTo: 'origin'
@registerTask 'default', ['test']
@registerTask 'build', ['clean', 'jshint', 'browserify', 'uglify']
@registerTask 'build', ['clean', 'jshint', 'jscs', 'browserify', 'uglify']
@registerTask 'test', ['build', 'mochacli']

View File

@@ -1,6 +1,6 @@
{
"name": "pofile",
"version": "0.2.7",
"version": "1.0.1",
"authors": [
"Ruben Vermeersch <ruben@rocketeer.be>"
],

220
dist/pofile.js vendored
View File

@@ -1,6 +1,6 @@
require=(function e(t,n,r){function s(o,u){if(!n[o]){if(!t[o]){var a=typeof require=="function"&&require;if(!u&&a)return a(o,!0);if(i)return i(o,!0);throw new Error("Cannot find module '"+o+"'")}var f=n[o]={exports:{}};t[o][0].call(f.exports,function(e){var n=t[o][1][e];return s(n?n:e)},f,f.exports,e,t,n,r)}return n[o].exports}var i=typeof require=="function"&&require;for(var o=0;o<r.length;o++)s(r[o]);return s})({"W8CkM0":[function(require,module,exports){
var fs = require('fs'),
isArray = require('lodash.isarray');
var fs = require('fs');
var isArray = require('lodash.isarray');
function trim(string) {
return string.replace(/^\s+|\s+$/g, '');
@@ -8,6 +8,7 @@ function trim(string) {
var PO = function () {
this.comments = [];
this.extractedComments = [];
this.headers = {};
this.items = [];
};
@@ -17,21 +18,26 @@ PO.prototype.save = function (filename, callback) {
};
PO.prototype.toString = function () {
var lines = [],
that = this;
var lines = [];
if (this.comments) {
this.comments.forEach(function (comment) {
lines.push('# ' + comment);
});
}
if (this.extractedComments) {
this.extractedComments.forEach(function (comment) {
lines.push('#. ' + comment);
});
}
lines.push('msgid ""');
lines.push('msgstr ""');
var keys = Object.keys(this.headers);
var self = this;
keys.forEach(function (key) {
lines.push('"' + key + ': ' + that.headers[key] + '\\n"');
lines.push('"' + key + ': ' + self.headers[key] + '\\n"');
});
lines.push('');
@@ -41,7 +47,7 @@ PO.prototype.toString = function () {
lines.push('');
});
return lines.join("\n");
return lines.join('\n');
};
PO.load = function (filename, callback) {
@@ -57,10 +63,15 @@ PO.load = function (filename, callback) {
PO.parse = function (data) {
//support both unix and windows newline formats.
data = data.replace(/\r\n/g, '\n');
var po = new PO(),
sections = data.split(/\n\n/),
headers = sections.shift(),
lines = sections.join("\n").split(/\n/);
var po = new PO();
var sections = data.split(/\n\n/);
var headers = [sections.shift()];
//everything until the first 'msgid ""' is considered header
while (headers[headers.length - 1].indexOf('msgid ""') < 0) {
headers.push(sections.shift());
}
headers = headers.join('\n');
var lines = sections.join('\n').split(/\n/);
po.headers = {
'Project-Id-Version': '',
@@ -75,103 +86,145 @@ PO.parse = function (data) {
'Plural-Forms': '',
};
headers.split(/\n/).forEach(function (header) {
if (header.match(/^#/)) {
po.comments.push(header.replace(/^#\s*/, ''));
headers.split(/\n/).reduce(function (acc, line) {
if (acc.merge) {
//join lines, remove last resp. first "
line = acc.pop().slice(0, -1) + line.slice(1);
delete acc.merge;
}
if (header.match(/^"/)) {
if (/^".*"$/.test(line) && !/^".*\\n"$/.test(line)) {
acc.merge = true;
}
acc.push(line);
return acc;
}, []).forEach(function (header) {
if (header.match(/^#\./)) {
po.extractedComments.push(header.replace(/^#\.\s*/, ''));
} else if (header.match(/^#/)) {
po.comments.push(header.replace(/^#\s*/, ''));
} else if (header.match(/^"/)) {
header = header.trim().replace(/^"/, '').replace(/\\n"$/, '');
var p = header.split(/:/),
name = p.shift().trim(),
value = p.join(':').trim();
var p = header.split(/:/);
var name = p.shift().trim();
var value = p.join(':').trim();
po.headers[name] = value;
}
});
var item = new PO.Item(),
context = null,
plural = 0,
obsolete = false;
var item = new PO.Item();
var context = null;
var plural = 0;
var obsoleteCount = 0;
var noCommentLineCount = 0;
function finish() {
if (item.msgid.length > 0) {
if (obsoleteCount >= noCommentLineCount) {
item.obsolete = true;
}
obsoleteCount = 0;
noCommentLineCount = 0;
po.items.push(item);
item = new PO.Item();
item.obsolete = obsolete;
obsolete = false;
}
}
function extract(string) {
string = trim(string);
string = string.replace(/^[^"]*"|"$/g, '');
string = string.replace(/\\"/g, '"');
string = string.replace(/\\\\/g, '\\');
string = string.replace(/\\([abtnvfr'"\\?]|([0-7]{3})|x([0-9a-fA-F]{2}))/g, function (match, esc, oct, hex) {
if (oct) {
return String.fromCharCode(parseInt(oct, 8));
}
if (hex) {
return String.fromCharCode(parseInt(hex, 16));
}
switch (esc) {
case 'a':
return '\x07';
case 'b':
return '\b';
case 't':
return '\t';
case 'n':
return '\n';
case 'v':
return '\v';
case 'f':
return '\f';
case 'r':
return '\r';
default:
return esc;
}
});
return string;
}
while (lines.length > 0) {
var line = trim(lines.shift()),
add = false;
var line = trim(lines.shift());
var lineObsolete = false;
var add = false;
if (line.match(/^#\~/)) { // Obsolete item
obsolete = true;
//only remove the obsolte comment mark, here
//might be, this is a new item, so
//only remember, this line is marked obsolete, count after line is parsed
line = trim(line.substring(2));
} else {
obsolete = false;
lineObsolete = true;
}
if (line.match(/^#:/)) { // Reference
finish();
item.references.push(trim(line.replace(/^#:/, '')));
}
else if (line.match(/^#,/)) { // Flags
} else if (line.match(/^#,/)) { // Flags
finish();
var flags = trim(line.replace(/^#,/, '')).split(",");
var flags = trim(line.replace(/^#,/, '')).split(',');
for (var i = 0; i < flags.length; i++) {
item.flags[flags[i]] = true;
}
}
else if (line.match(/^#\s+/)) { // Translator comment
} else if (line.match(/^#($|\s+)/)) { // Translator comment
finish();
item.comments.push(trim(line.replace(/^#\s+/, '')));
}
else if (line.match(/^#\./)) { // Extracted comment
item.comments.push(trim(line.replace(/^#($|\s+)/, '')));
} else if (line.match(/^#\./)) { // Extracted comment
finish();
item.extractedComments.push(trim(line.replace(/^#\./, '')));
}
else if (line.match(/^msgid_plural/)) { // Plural form
} else if (line.match(/^msgid_plural/)) { // Plural form
item.msgid_plural = extract(line);
context = 'msgid_plural';
}
else if (line.match(/^msgid/)) { // Original
noCommentLineCount++;
} else if (line.match(/^msgid/)) { // Original
finish();
item.msgid = extract(line);
context = 'msgid';
}
else if (line.match(/^msgstr/)) { // Translation
noCommentLineCount++;
} else if (line.match(/^msgstr/)) { // Translation
var m = line.match(/^msgstr\[(\d+)\]/);
plural = m && m[1] ? parseInt(m[1]) : 0;
item.msgstr[plural] = extract(line);
context = 'msgstr';
}
else if (line.match(/^msgctxt/)) { // Context
noCommentLineCount++;
} else if (line.match(/^msgctxt/)) { // Context
finish();
item.msgctxt = extract(line);
}
else { // Probably multiline string or blank
noCommentLineCount++;
} else { // Probably multiline string or blank
if (line.length > 0) {
noCommentLineCount++;
if (context === 'msgstr') {
item.msgstr[plural] += extract(line);
}
else if (context === 'msgid') {
} else if (context === 'msgid') {
item.msgid += extract(line);
}
else if (context === 'msgid_plural') {
} else if (context === 'msgid_plural') {
item.msgid_plural += extract(line);
}
}
}
if (lineObsolete) {
// Count obsolete lines for this item
obsoleteCount++;
}
}
finish();
@@ -191,26 +244,44 @@ PO.Item = function () {
};
PO.Item.prototype.toString = function () {
var lines = [],
that = this;
var lines = [];
var self = this;
// reverse what extract(string) method during PO.parse does
var _escape = function (string) {
string = string.replace(/\\/g, '\\\\');
return string.replace(/"/g, '\\"');
// don't unescape \n, since string can never contain it
// since split('\n') is called on it
string = string.replace(/[\x07\b\t\v\f\r"\\]/g, function (match) {
switch (match) {
case '\x07':
return '\\a';
case '\b':
return '\\b';
case '\t':
return '\\t';
case '\v':
return '\\v';
case '\f':
return '\\f';
case '\r':
return '\\r';
default:
return '\\' + match;
}
});
return string;
};
var _process = function (keyword, text, i) {
var lines = [],
parts = text.split(/\n/),
index = typeof i !== 'undefined' ? '[' + i + ']' : '';
var lines = [];
var parts = text.split(/\n/);
var index = typeof i !== 'undefined' ? '[' + i + ']' : '';
if (parts.length > 1) {
lines.push(keyword + index + ' ""');
parts.forEach(function (part) {
lines.push('"' + _escape(part) + '"');
});
}
else {
} else {
lines.push(keyword + index + ' "' + _escape(text) + '"');
}
return lines;
@@ -231,31 +302,34 @@ PO.Item.prototype.toString = function () {
lines.push('#: ' + ref);
});
var flags = Object.keys(this.flags);
var flags = Object.keys(this.flags).filter(function (flag) {
return !!this.flags[flag];
}, this);
if (flags.length > 0) {
lines.push('#, ' + flags.join(","));
lines.push('#, ' + flags.join(','));
}
var mkObsolete = this.obsolete ? '#~ ' : '';
['msgctxt', 'msgid', 'msgid_plural', 'msgstr'].forEach(function (keyword) {
var text = that[keyword];
var text = self[keyword];
if (text != null) {
if (isArray(text) && text.length > 1) {
text.forEach(function (t, i) {
lines = lines.concat(_process(keyword, t, i));
lines = lines.concat(mkObsolete + _process(keyword, t, i));
});
}
else {
} else {
text = isArray(text) ? text.join() : text;
lines = lines.concat(_process(keyword, text));
var processed = _process(keyword, text);
//handle \n in single-line texts (can not be handled in _escape)
for (var i = 1; i < processed.length - 1; i++) {
processed[i] = processed[i].slice(0, -1) + '\\n"';
}
lines = lines.concat(mkObsolete + processed.join('\n' + mkObsolete));
}
}
});
if (this.obsolete) {
return "#~ " + lines.join("\n#~ ");
} else {
return lines.join("\n");
}
return lines.join('\n');
};
module.exports = PO;

2
dist/pofile.min.js vendored

File diff suppressed because one or more lines are too long

220
lib/po.js
View File

@@ -1,5 +1,5 @@
var fs = require('fs'),
isArray = require('lodash.isarray');
var fs = require('fs');
var isArray = require('lodash.isarray');
function trim(string) {
return string.replace(/^\s+|\s+$/g, '');
@@ -7,6 +7,7 @@ function trim(string) {
var PO = function () {
this.comments = [];
this.extractedComments = [];
this.headers = {};
this.items = [];
};
@@ -16,21 +17,26 @@ PO.prototype.save = function (filename, callback) {
};
PO.prototype.toString = function () {
var lines = [],
that = this;
var lines = [];
if (this.comments) {
this.comments.forEach(function (comment) {
lines.push('# ' + comment);
});
}
if (this.extractedComments) {
this.extractedComments.forEach(function (comment) {
lines.push('#. ' + comment);
});
}
lines.push('msgid ""');
lines.push('msgstr ""');
var keys = Object.keys(this.headers);
var self = this;
keys.forEach(function (key) {
lines.push('"' + key + ': ' + that.headers[key] + '\\n"');
lines.push('"' + key + ': ' + self.headers[key] + '\\n"');
});
lines.push('');
@@ -40,7 +46,7 @@ PO.prototype.toString = function () {
lines.push('');
});
return lines.join("\n");
return lines.join('\n');
};
PO.load = function (filename, callback) {
@@ -56,10 +62,15 @@ PO.load = function (filename, callback) {
PO.parse = function (data) {
//support both unix and windows newline formats.
data = data.replace(/\r\n/g, '\n');
var po = new PO(),
sections = data.split(/\n\n/),
headers = sections.shift(),
lines = sections.join("\n").split(/\n/);
var po = new PO();
var sections = data.split(/\n\n/);
var headers = [sections.shift()];
//everything until the first 'msgid ""' is considered header
while (headers[headers.length - 1].indexOf('msgid ""') < 0) {
headers.push(sections.shift());
}
headers = headers.join('\n');
var lines = sections.join('\n').split(/\n/);
po.headers = {
'Project-Id-Version': '',
@@ -74,103 +85,145 @@ PO.parse = function (data) {
'Plural-Forms': '',
};
headers.split(/\n/).forEach(function (header) {
if (header.match(/^#/)) {
po.comments.push(header.replace(/^#\s*/, ''));
headers.split(/\n/).reduce(function (acc, line) {
if (acc.merge) {
//join lines, remove last resp. first "
line = acc.pop().slice(0, -1) + line.slice(1);
delete acc.merge;
}
if (header.match(/^"/)) {
if (/^".*"$/.test(line) && !/^".*\\n"$/.test(line)) {
acc.merge = true;
}
acc.push(line);
return acc;
}, []).forEach(function (header) {
if (header.match(/^#\./)) {
po.extractedComments.push(header.replace(/^#\.\s*/, ''));
} else if (header.match(/^#/)) {
po.comments.push(header.replace(/^#\s*/, ''));
} else if (header.match(/^"/)) {
header = header.trim().replace(/^"/, '').replace(/\\n"$/, '');
var p = header.split(/:/),
name = p.shift().trim(),
value = p.join(':').trim();
var p = header.split(/:/);
var name = p.shift().trim();
var value = p.join(':').trim();
po.headers[name] = value;
}
});
var item = new PO.Item(),
context = null,
plural = 0,
obsolete = false;
var item = new PO.Item();
var context = null;
var plural = 0;
var obsoleteCount = 0;
var noCommentLineCount = 0;
function finish() {
if (item.msgid.length > 0) {
if (obsoleteCount >= noCommentLineCount) {
item.obsolete = true;
}
obsoleteCount = 0;
noCommentLineCount = 0;
po.items.push(item);
item = new PO.Item();
item.obsolete = obsolete;
obsolete = false;
}
}
function extract(string) {
string = trim(string);
string = string.replace(/^[^"]*"|"$/g, '');
string = string.replace(/\\"/g, '"');
string = string.replace(/\\\\/g, '\\');
string = string.replace(/\\([abtnvfr'"\\?]|([0-7]{3})|x([0-9a-fA-F]{2}))/g, function (match, esc, oct, hex) {
if (oct) {
return String.fromCharCode(parseInt(oct, 8));
}
if (hex) {
return String.fromCharCode(parseInt(hex, 16));
}
switch (esc) {
case 'a':
return '\x07';
case 'b':
return '\b';
case 't':
return '\t';
case 'n':
return '\n';
case 'v':
return '\v';
case 'f':
return '\f';
case 'r':
return '\r';
default:
return esc;
}
});
return string;
}
while (lines.length > 0) {
var line = trim(lines.shift()),
add = false;
var line = trim(lines.shift());
var lineObsolete = false;
var add = false;
if (line.match(/^#\~/)) { // Obsolete item
obsolete = true;
//only remove the obsolte comment mark, here
//might be, this is a new item, so
//only remember, this line is marked obsolete, count after line is parsed
line = trim(line.substring(2));
} else {
obsolete = false;
lineObsolete = true;
}
if (line.match(/^#:/)) { // Reference
finish();
item.references.push(trim(line.replace(/^#:/, '')));
}
else if (line.match(/^#,/)) { // Flags
} else if (line.match(/^#,/)) { // Flags
finish();
var flags = trim(line.replace(/^#,/, '')).split(",");
var flags = trim(line.replace(/^#,/, '')).split(',');
for (var i = 0; i < flags.length; i++) {
item.flags[flags[i]] = true;
}
}
else if (line.match(/^#\s+/)) { // Translator comment
} else if (line.match(/^#($|\s+)/)) { // Translator comment
finish();
item.comments.push(trim(line.replace(/^#\s+/, '')));
}
else if (line.match(/^#\./)) { // Extracted comment
item.comments.push(trim(line.replace(/^#($|\s+)/, '')));
} else if (line.match(/^#\./)) { // Extracted comment
finish();
item.extractedComments.push(trim(line.replace(/^#\./, '')));
}
else if (line.match(/^msgid_plural/)) { // Plural form
} else if (line.match(/^msgid_plural/)) { // Plural form
item.msgid_plural = extract(line);
context = 'msgid_plural';
}
else if (line.match(/^msgid/)) { // Original
noCommentLineCount++;
} else if (line.match(/^msgid/)) { // Original
finish();
item.msgid = extract(line);
context = 'msgid';
}
else if (line.match(/^msgstr/)) { // Translation
noCommentLineCount++;
} else if (line.match(/^msgstr/)) { // Translation
var m = line.match(/^msgstr\[(\d+)\]/);
plural = m && m[1] ? parseInt(m[1]) : 0;
item.msgstr[plural] = extract(line);
context = 'msgstr';
}
else if (line.match(/^msgctxt/)) { // Context
noCommentLineCount++;
} else if (line.match(/^msgctxt/)) { // Context
finish();
item.msgctxt = extract(line);
}
else { // Probably multiline string or blank
noCommentLineCount++;
} else { // Probably multiline string or blank
if (line.length > 0) {
noCommentLineCount++;
if (context === 'msgstr') {
item.msgstr[plural] += extract(line);
}
else if (context === 'msgid') {
} else if (context === 'msgid') {
item.msgid += extract(line);
}
else if (context === 'msgid_plural') {
} else if (context === 'msgid_plural') {
item.msgid_plural += extract(line);
}
}
}
if (lineObsolete) {
// Count obsolete lines for this item
obsoleteCount++;
}
}
finish();
@@ -190,26 +243,44 @@ PO.Item = function () {
};
PO.Item.prototype.toString = function () {
var lines = [],
that = this;
var lines = [];
var self = this;
// reverse what extract(string) method during PO.parse does
var _escape = function (string) {
string = string.replace(/\\/g, '\\\\');
return string.replace(/"/g, '\\"');
// don't unescape \n, since string can never contain it
// since split('\n') is called on it
string = string.replace(/[\x07\b\t\v\f\r"\\]/g, function (match) {
switch (match) {
case '\x07':
return '\\a';
case '\b':
return '\\b';
case '\t':
return '\\t';
case '\v':
return '\\v';
case '\f':
return '\\f';
case '\r':
return '\\r';
default:
return '\\' + match;
}
});
return string;
};
var _process = function (keyword, text, i) {
var lines = [],
parts = text.split(/\n/),
index = typeof i !== 'undefined' ? '[' + i + ']' : '';
var lines = [];
var parts = text.split(/\n/);
var index = typeof i !== 'undefined' ? '[' + i + ']' : '';
if (parts.length > 1) {
lines.push(keyword + index + ' ""');
parts.forEach(function (part) {
lines.push('"' + _escape(part) + '"');
});
}
else {
} else {
lines.push(keyword + index + ' "' + _escape(text) + '"');
}
return lines;
@@ -230,31 +301,34 @@ PO.Item.prototype.toString = function () {
lines.push('#: ' + ref);
});
var flags = Object.keys(this.flags);
var flags = Object.keys(this.flags).filter(function (flag) {
return !!this.flags[flag];
}, this);
if (flags.length > 0) {
lines.push('#, ' + flags.join(","));
lines.push('#, ' + flags.join(','));
}
var mkObsolete = this.obsolete ? '#~ ' : '';
['msgctxt', 'msgid', 'msgid_plural', 'msgstr'].forEach(function (keyword) {
var text = that[keyword];
var text = self[keyword];
if (text != null) {
if (isArray(text) && text.length > 1) {
text.forEach(function (t, i) {
lines = lines.concat(_process(keyword, t, i));
lines = lines.concat(mkObsolete + _process(keyword, t, i));
});
}
else {
} else {
text = isArray(text) ? text.join() : text;
lines = lines.concat(_process(keyword, text));
var processed = _process(keyword, text);
//handle \n in single-line texts (can not be handled in _escape)
for (var i = 1; i < processed.length - 1; i++) {
processed[i] = processed[i].slice(0, -1) + '\\n"';
}
lines = lines.concat(mkObsolete + processed.join('\n' + mkObsolete));
}
}
});
if (this.obsolete) {
return "#~ " + lines.join("\n#~ ");
} else {
return lines.join("\n");
}
return lines.join('\n');
};
module.exports = PO;

View File

@@ -1,7 +1,7 @@
{
"name": "pofile",
"description": "Parse and serialize Gettext PO files.",
"version": "0.2.7",
"version": "1.0.1",
"author": {
"name": "Ruben Vermeersch",
"email": "ruben@savanne.be",
@@ -35,15 +35,16 @@
"test": "test"
},
"devDependencies": {
"grunt": "~0.4.2",
"grunt-contrib-watch": "~0.5.3",
"grunt-contrib-jshint": "~0.7.2",
"grunt-mocha-cli": "~1.4.0",
"grunt-contrib-uglify": "~0.2.7",
"browserify": "~3.11.1",
"grunt": "~0.4.2",
"grunt-browserify": "~1.3.0",
"grunt-bump": "0.0.13",
"grunt-contrib-clean": "~0.5.0",
"grunt-bump": "0.0.13"
"grunt-contrib-jshint": "~0.7.2",
"grunt-contrib-uglify": "~0.2.7",
"grunt-contrib-watch": "~0.5.3",
"grunt-jscs-checker": "^0.5.1",
"grunt-mocha-cli": "~1.4.0"
},
"dependencies": {
"lodash.isarray": "~2.4.1"

View File

@@ -1,4 +1,5 @@
# French translation of Link (6.x-2.9)
# Copyright (c) 2011 by the French translation team
#
msgid ""

View File

@@ -17,3 +17,43 @@ msgstr ""
msgid "The name field must not contain characters like \" or \\"
msgstr ""
# possibility to reorder items depending on locale
#. Format of addresses
#. %1$s is the street
#. %2$s is the postal code
#. %3$s is the city
#. %4$s is the state
#. %5$s is the country
msgid ""
"%1$s\n"
"%2$s %3$s\n"
"%4$s\n"
"%5$s"
msgstr ""
# "i18"ned code
#. used in <pre> environment, so don't remove any control sequences
msgid ""
"define('some/test/module', function () {\n"
"\t'use strict';\n"
"\treturn {};\n"
"});\n"
""
msgstr ""
"define('random/test/file', function () {\n"
"\t'use strict';\n"
"\treturn {};\n"
"});\n"
""
# all one-letter escape characters
# be aware, that \a, \b, \v, \f and \r should not be used
# in i18ned messages (according to gettext tools)
# however, they should be properly parsed, anyway
msgid ""
"\a\b\t\n"
"\v\f\r"
msgstr ""
"\a\b\t\n"
"\v\f\r"

View File

@@ -1,6 +1,7 @@
# French translation of Link (6.x-2.9)
# Copyright (c) 2011 by the French translation team
#
#. extracted from test
msgid ""
msgstr ""
"Project-Id-Version: Link (6.x-2.9)\n"
@@ -19,3 +20,11 @@ msgstr ""
#. Extracted comment
msgid "Title, as plain text"
msgstr "Attribut title, en tant que texte brut"
#
#.
#:
#, fuzzy
msgid "Empty comment"
msgstr "Empty"

View File

@@ -21,3 +21,13 @@ msgstr "{{dataLoader.data.length}} resultaten"
#~ msgid "Add order"
#~ msgstr "Order toevoegen"
#~ # commented obsolete item
#~ #, fuzzy
#~ msgid "Commented item"
#~ msgstr "not sure"
# commented obsolete item
#, fuzzy
#~ msgid "Second commented item"
#~ msgstr "also not sure"

View File

@@ -1,6 +1,8 @@
# French translation of Link (6.x-2.9)
# Copyright (c) 2011 by the French translation team
#
## Plural-Forms by polish translation team to demonstrate multi-line ##
#
msgid ""
msgstr ""
"Project-Id-Version: Link (6.x-2.9)\n"
@@ -10,7 +12,8 @@ msgstr ""
"MIME-Version: 1.0\n"
"Content-Type: text/plain; charset=UTF-8\n"
"Content-Transfer-Encoding: 8bit\n"
"Plural-Forms: nplurals=2; plural=(n > 1);\n"
"Plural-Forms: nplurals=3; plural=n==1 ? 0 : n%10>=2 && n%10<=4 && (n%100<10 "
"|| n%100>=20) ? 1 : 2;\n"
"Last-Translator: Ruben Vermeersch <ruben@rocketeer.be>\n"
"Language: fr\n"
"X-Generator: Poedit 1.6.2\n"

View File

@@ -24,3 +24,7 @@ msgstr "Attribut title, en tant que texte brut"
#: b
msgid "X"
msgstr "Y"
#: standard input:12 standard input:17
msgid "Z"
msgstr "ZZ"

View File

@@ -7,59 +7,95 @@ describe('Parse', function () {
var po = PO.parse(fs.readFileSync(__dirname + '/fixtures/big.po', 'utf8'));
assert.notEqual(po, null);
assert.equal(po.items.length, 69);
var item = po.items[0];
assert.equal(item.msgid, "Title");
assert.equal(item.msgstr, "Titre");
assert.equal(item.msgid, 'Title');
assert.equal(item.msgstr, 'Titre');
});
it('Handles multi-line strings', function () {
var po = PO.parse(fs.readFileSync(__dirname + '/fixtures/multi-line.po', 'utf8'));
assert.notEqual(po, null);
assert.equal(po.items.length, 1);
var item = po.items[0];
assert.equal(item.msgid, "The following placeholder tokens can be used in both paths and titles. When used in a path or title, they will be replaced with the appropriate values.");
assert.equal(item.msgstr, "Les ébauches de jetons suivantes peuvent être utilisées à la fois dans les chemins et dans les titres. Lorsqu'elles sont utilisées dans un chemin ou un titre, elles seront remplacées par les valeurs appropriées.");
assert.equal(item.msgid, 'The following placeholder tokens can be used in both paths and titles. When used in a path or title, they will be replaced with the appropriate values.');
assert.equal(item.msgstr, 'Les ébauches de jetons suivantes peuvent être utilisées à la fois dans les chemins et dans les titres. Lorsqu\'elles sont utilisées dans un chemin ou un titre, elles seront remplacées par les valeurs appropriées.');
});
it('Handles multi-line headers', function () {
var po = PO.parse(fs.readFileSync(__dirname + '/fixtures/multi-line.po', 'utf8'));
assert.notEqual(po, null);
assert.equal(po.items.length, 1);
assert.equal(po.headers['Plural-Forms'], 'nplurals=3; plural=n==1 ? 0 : n%10>=2 && n%10<=4 && (n%100<10 || n%100>=20) ? 1 : 2;');
});
it('Handle empty comments', function (done) {
PO.load(__dirname + '/fixtures/comment.po', function (err, po) {
assert.equal(err, null);
var item = po.items[1];
assert.equal(item.msgid, 'Empty comment');
assert.equal(item.msgstr, 'Empty');
assert.deepEqual(item.comments, ['']);
assert.deepEqual(item.extractedComments, ['']);
assert.deepEqual(item.references, ['']);
done();
});
});
it('Handles translator comments', function () {
var po = PO.parse(fs.readFileSync(__dirname + '/fixtures/comment.po', 'utf8'));
assert.notEqual(po, null);
assert.equal(po.items.length, 1);
assert.equal(po.items.length, 2);
var item = po.items[0];
assert.equal(item.msgid, "Title, as plain text");
assert.equal(item.msgstr, "Attribut title, en tant que texte brut");
assert.deepEqual(item.comments, ["Translator comment"]);
assert.equal(item.msgid, 'Title, as plain text');
assert.equal(item.msgstr, 'Attribut title, en tant que texte brut');
assert.deepEqual(item.comments, ['Translator comment']);
});
it('Handles extracted comments', function () {
var po = PO.parse(fs.readFileSync(__dirname + '/fixtures/comment.po', 'utf8'));
assert.notEqual(po, null);
assert.equal(po.items.length, 1);
assert.equal(po.items.length, 2);
assert.equal(po.extractedComments.length, 1);
assert.equal(po.extractedComments[0], 'extracted from test');
var item = po.items[0];
assert.equal(item.msgid, "Title, as plain text");
assert.equal(item.msgstr, "Attribut title, en tant que texte brut");
assert.deepEqual(item.extractedComments, ["Extracted comment"]);
assert.equal(item.msgid, 'Title, as plain text');
assert.equal(item.msgstr, 'Attribut title, en tant que texte brut');
assert.deepEqual(item.extractedComments, ['Extracted comment']);
});
it('Handles string references', function () {
describe('Handles string references', function () {
var po = PO.parse(fs.readFileSync(__dirname + '/fixtures/reference.po', 'utf8'));
assert.notEqual(po, null);
assert.equal(po.items.length, 2);
var item = po.items[0];
assert.equal(item.msgid, "Title, as plain text");
assert.equal(item.msgstr, "Attribut title, en tant que texte brut");
assert.deepEqual(item.comments, ["Comment"]);
assert.deepEqual(item.references, [".tmp/crm/controllers/map.js"]);
item = po.items[1];
assert.equal(item.msgid, "X");
assert.equal(item.msgstr, "Y");
assert.deepEqual(item.references, ["a", "b"]);
assert.equal(po.items.length, 3);
it('in simple cases', function () {
var item = po.items[0];
assert.equal(item.msgid, 'Title, as plain text');
assert.equal(item.msgstr, 'Attribut title, en tant que texte brut');
assert.deepEqual(item.comments, ['Comment']);
assert.deepEqual(item.references, ['.tmp/crm/controllers/map.js']);
});
it('with two different references', function () {
var item = po.items[1];
assert.equal(item.msgid, 'X');
assert.equal(item.msgstr, 'Y');
assert.deepEqual(item.references, ['a', 'b']);
});
it('and does not process reference items', function () {
var item = po.items[2];
assert.equal(item.msgid, 'Z');
assert.equal(item.msgstr, 'ZZ');
assert.deepEqual(item.references, ['standard input:12 standard input:17']);
});
});
it('Parses flags', function () {
@@ -68,8 +104,8 @@ describe('Parse', function () {
assert.equal(po.items.length, 1);
var item = po.items[0];
assert.equal(item.msgid, "Sources");
assert.equal(item.msgstr, "Source");
assert.equal(item.msgid, 'Sources');
assert.equal(item.msgstr, 'Source');
assert.notEqual(item.flags, null);
assert.equal(item.flags.fuzzy, true);
});
@@ -88,32 +124,52 @@ describe('Parse', function () {
it('Handles obsolete items', function () {
var po = PO.parse(fs.readFileSync(__dirname + '/fixtures/commented.po', 'utf8'));
assert.equal(po.items.length, 2);
assert.equal(po.items.length, 4);
var item = po.items[0];
assert.equal(item.obsolete, false);
assert.equal(item.msgid, "{{dataLoader.data.length}} results");
assert.equal(item.msgstr, "{{dataLoader.data.length}} resultaten");
assert.equal(item.msgid, '{{dataLoader.data.length}} results');
assert.equal(item.msgstr, '{{dataLoader.data.length}} resultaten');
item = po.items[1];
assert.equal(item.obsolete, true);
assert.equal(item.msgid, "Add order");
assert.equal(item.msgstr, "Order toevoegen");
assert.equal(item.msgid, 'Add order');
assert.equal(item.msgstr, 'Order toevoegen');
item = po.items[2];
assert.equal(item.obsolete, true);
assert.equal(item.msgid, 'Commented item');
assert.equal(item.msgstr, 'not sure');
item = po.items[3];
assert.equal(item.obsolete, true);
assert.equal(item.msgid, 'Second commented item');
assert.equal(item.msgstr, 'also not sure');
});
describe('C-Strings', function () {
var po = PO.parse(fs.readFileSync(__dirname + '/fixtures/c-strings.po', 'utf8'));
it('should parse the c-strings.po file', function () {
var po = PO.parse(fs.readFileSync(__dirname + '/fixtures/c-strings.po', 'utf8'));
assert.notEqual(po, null);
});
it('should extract strings containing " and \\ characters', function () {
var po = PO.parse(fs.readFileSync(__dirname + '/fixtures/c-strings.po', 'utf8'));
var items = po.items.filter(function (item) {
return (/^The name field must not contain/).test(item.msgid);
});
assert.equal(items[0].msgid, 'The name field must not contain characters like " or \\');
});
it('should handle \\n characters', function () {
var item = po.items[1];
assert.equal(item.msgid, '%1$s\n%2$s %3$s\n%4$s\n%5$s');
});
it('should handle \\t characters', function () {
var item = po.items[2];
assert.equal(item.msgid, 'define(\'some/test/module\', function () {\n' +
'\t\'use strict\';\n' +
'\treturn {};\n' +
'});\n');
});
});
});

View File

@@ -3,7 +3,7 @@ var fs = require('fs');
var PO = require('..');
function assertHasLine(str, line) {
var lines = str.split("\n");
var lines = str.split('\n');
var found = false;
for (var i = 0; i < lines.length; i++) {
@@ -13,7 +13,21 @@ function assertHasLine(str, line) {
}
}
assert(found, "Could not find line: " + line);
assert(found, 'Could not find line: ' + line);
}
function assertDoesntHaveLine(str, line) {
var lines = str.split('\n');
var found = false;
for (var i = 0; i < lines.length; i++) {
if (lines[i].trim() === line) {
found = true;
break;
}
}
assert(!found, 'Shouldn\'t have line: ' + line);
}
describe('Write', function () {
@@ -21,34 +35,46 @@ describe('Write', function () {
var input = fs.readFileSync(__dirname + '/fixtures/fuzzy.po', 'utf8');
var po = PO.parse(input);
var str = po.toString();
assertHasLine(str, "#, fuzzy");
assertHasLine(str, '#, fuzzy');
});
it('write flags only when true', function () {
var input = fs.readFileSync(__dirname + '/fixtures/fuzzy.po', 'utf8');
var po = PO.parse(input);
// Flip flag
po.items[0].flags.fuzzy = false;
var str = po.toString();
assertDoesntHaveLine(str, '#, fuzzy');
});
it('write msgid', function () {
var input = fs.readFileSync(__dirname + '/fixtures/fuzzy.po', 'utf8');
var po = PO.parse(input);
var str = po.toString();
assertHasLine(str, "msgid \"Sources\"");
assertHasLine(str, 'msgid "Sources"');
});
it('write msgstr', function () {
var input = fs.readFileSync(__dirname + '/fixtures/fuzzy.po', 'utf8');
var po = PO.parse(input);
var str = po.toString();
assertHasLine(str, "msgstr \"Source\"");
assertHasLine(str, 'msgstr "Source"');
});
it('write translator comment', function () {
var input = fs.readFileSync(__dirname + '/fixtures/comment.po', 'utf8');
var po = PO.parse(input);
var str = po.toString();
assertHasLine(str, "# Translator comment");
assertHasLine(str, '# Translator comment');
});
it('write extracted comment', function () {
var input = fs.readFileSync(__dirname + '/fixtures/comment.po', 'utf8');
var po = PO.parse(input);
var str = po.toString();
assertHasLine(str, '#. extracted from test');
assertHasLine(str, '#. Extracted comment');
});
@@ -61,6 +87,22 @@ describe('Write', function () {
assertHasLine(str, '#~ msgstr "Order toevoegen"');
});
it('write obsolete items with comment', function () {
var input = fs.readFileSync(__dirname + '/fixtures/commented.po', 'utf8');
var po = PO.parse(input);
var str = po.toString();
//this is what msgcat tool of gettext does: don't print #~ for comments
assertHasLine(str, '# commented obsolete item');
assertHasLine(str, '#, fuzzy');
//items made obsolete by commenting every keyword with #~
assertHasLine(str, '#~ msgid "Commented item"');
assertHasLine(str, '#~ msgstr "not sure"');
assertHasLine(str, '#~ msgid "Second commented item"');
assertHasLine(str, '#~ msgstr "also not sure"');
});
describe('C-Strings', function () {
it('should escape "', function () {
var item = new PO.Item();
@@ -76,6 +118,15 @@ describe('Write', function () {
assertHasLine(item.toString(), 'msgid "\\\\ should be written escaped"');
});
it('should escape \\n', function () {
var item = new PO.Item();
item.msgid = '\n should be written escaped';
assertHasLine(item.toString(), 'msgid ""');
assertHasLine(item.toString(), '"\\n"');
assertHasLine(item.toString(), '" should be written escaped"');
});
it('should write identical file after parsing a file', function () {
var input = fs.readFileSync(__dirname + '/fixtures/c-strings.po', 'utf8');
var po = PO.parse(input);