#6610 Convert trac syntax better

v1.1.0
closed
Tim Van Steenburgh
trac (14) import (116) sf-2 (994)
General
Dave Brondsema
2015-08-20
2013-08-27
Dave Brondsema
No

The Trac ticket export/import process doesn't handle the following syntax correctly:

  • headings e.g. == NOTE ==
  • preformatted blocks, using {{{ ... }}} -- this does seem to come through OK in comments, presumably because we're converting those differently
  • lists that have no blank line preceding them, e.g. blah blah\n* foo\n* bar
  • https://sourceforge.net/apps/trac/gfarm/ticket/594#comment:5 gets a \n incorrectly introduced between "merged into" and "2.6 branch"
  • ---- for a horizontal line becomes a markdown heading if there is no blank line before it
  • special characters that Markdown might interpret need to be escaped. Simple example: foo <a b c> bar Perhaps helpers.plain2markdown() will help.
  • Make sure external links like [http://google.com test] work (I'm not sure)

These are the issues I noticed - look through the Trac syntax reference to see if there are more.

Related

Tickets: #6140

Discussion

  • Dave Brondsema

    Dave Brondsema - 2013-08-27
    • summary: Convert trac heading syntax == --> Convert trac syntax better
    • Description has changed:

    Diff:

    --- old
+++ new
@@ -1 +1,12 @@
-Trac syntax for headings, e.g. `== NOTE ==` is not converted in the Trac ticket export/import process.
+The Trac ticket export/import process doesn't handle the following syntax correctly:
+
+* headings e.g. `== NOTE ==`
+* preformatted blocks, using `{{{ ... }}}` -- this does seem to come through OK in comments, presumably because we're converting those differently
+* lists that have no blank line preceding them, e.g. `blah blah\n* foo\n* bar`
+* https://sourceforge.net/apps/trac/gfarm/ticket/594#comment:5 gets a `\n` incorrectly introduced between "merged into" and "2.6 branch"
+* `----` for a horizontal line becomes a markdown heading if there is no blank line before it
+* special characters that Markdown might interpret need to be escaped.  Simple example: `foo <a b c> bar`  Perhaps helpers.plain2markdown() will help.
+* Make sure external links like `[http://google.com test]` work (I'm not sure) 
+
+
+These are the issues I noticed - look through the Trac syntax reference to see if there are more.
     

  • Dave Brondsema

    Dave Brondsema - 2013-09-09
    • Size: --> 2
     

  • Dave Brondsema

    Dave Brondsema - 2013-09-09

    If we use the same process used for comments (converting HTML to markdown, instead of using Trac syntax), we'll be consistent in how we handle Trac text and it'll be a lot easier to fix these issues, rather than parsing & converting syntaxes.

     

  • Dave Brondsema

    Dave Brondsema - 2013-09-20
    • Milestone: forge-backlog --> forge-oct-04
     

  • Tim Van Steenburgh - 2013-09-24
    • status: open --> in-progress
    • assigned_to: Tim Van Steenburgh
     

  • Dave Brondsema

    Dave Brondsema - 2013-10-04
    • Milestone: forge-oct-04 --> forge-oct-18
     

  • Tim Van Steenburgh - 2013-10-04
    • status: in-progress --> code-review
    • Milestone: forge-oct-18 --> forge-oct-04
     

  • Tim Van Steenburgh - 2013-10-04

    allura:tv/6610

    This is rebased to tv/6728, so that should be reviewed first.

    I did a fresh gfarm import here: https://sf-tvansteenburgh-1020.sb.sf.net/p/test6549/gfarm-formatted/

    Some tickets that show better formatting: 571, 612, 681, 594, 607

     

  • Tim Van Steenburgh - 2013-10-04
    • Milestone: forge-oct-04 --> forge-oct-18
     

  • Dave Brondsema

    Dave Brondsema - 2013-10-16
    • QA: Dave Brondsema
     

  • Dave Brondsema

    Dave Brondsema - 2013-10-16
    • status: code-review --> in-progress
     

  • Dave Brondsema

    Dave Brondsema - 2013-10-16

    There's over-escaping in http://sf-tvansteenburgh-1020.sb.sf.net/p/test6549/gfarm-formatted/287/ Or maybe it's linkifying of #123 ticket references, which ideally wouldn't happen within pre-formatted text (<code> and <pre>). html2text does that type of recognition (see its handle_data). Maybe we could hook into the escape_md_section call and only replace links there?

    Can "Originally posted by:" in comments be italicized like is done for ticket body?

     

  • Dave Brondsema

    Dave Brondsema - 2013-10-21
    • Milestone: forge-oct-18 --> forge-nov-01
     

  • Tim Van Steenburgh - 2013-10-23
    • status: in-progress --> code-review
     

  • Dave Brondsema

    Dave Brondsema - 2013-10-24

    You had mentioned (in private) a simple markdown parser/rewriter thing that you wrote for an earlier attempt on this ticket, and it might be good to re-use it for other things like [#6622]. Is that available somewhere?

     

    Related

    Tickets: #6622

  • Dave Brondsema

    Dave Brondsema - 2013-10-25
    • status: code-review --> in-progress
     

  • Dave Brondsema

    Dave Brondsema - 2013-10-25

    From the looks of it, the whole link_processing method should go away, for the same reasons you removed the short_link_ticket_pattern part of it; right? Moreover, link_processing is trac-specific so we don't want it in our generic import_support.py logic anyway.

     

  • Tim Van Steenburgh - 2013-10-30
    • status: in-progress --> code-review
     

  • Tim Van Steenburgh - 2013-10-30

    90e2f91..8d79813 tv/6610 -> tv/6610

    Refactored and confirmed that the remaining parsing is still needed.

     

  • Dave Brondsema

    Dave Brondsema - 2013-11-01
    • status: code-review --> closed
     

Log in to post a comment.