summaryrefslogtreecommitdiff
diff options
context:
space:
mode:
authorBehdad Esfahbod <behdad@behdad.org>2018-01-28 11:43:11 -0800
committerBehdad Esfahbod <behdad@behdad.org>2018-02-09 13:02:17 -0600
commit4ae5000b6633af8f2be9f2d5133e72c48b6360c5 (patch)
tree463e91e5ee72f70903959e71f4b1afc5cd7d98d8
parent544b56553b94a2425f52f4b6c29f0a9abe403161 (diff)
downloadfribidi-4ae5000b6633af8f2be9f2d5133e72c48b6360c5.tar.gz
Remove notes.org
-rw-r--r--notes.org318
1 files changed, 0 insertions, 318 deletions
diff --git a/notes.org b/notes.org
deleted file mode 100644
index 7a44ef6..0000000
--- a/notes.org
+++ /dev/null
@@ -1,318 +0,0 @@
-#+STARTUP: hidestars showall
-* <2014-03-08 Sat>
-** Running a test of the new features
-#+BEGIN_SRC sh
-cd test
-../bin/fribidi --charset CapRTL test_CapRTL_isolate.input
-#+END_SRC
- - No link to gen-bidi-type-tab .
-* <2014-03-16 Sun>
-** Running more extensive tests
-#+BEGIN_SRC sh
-../bin/fribidi --basedir --showinput --levels --charset CapRTL test_CapRTL_isolate.input
-#+END_SRC
-* <2015-05-24 Sun>
-** Issues not covered in my current implementation
- - In X2 and X3. Deal with overflow isolate count.
- - Terminology
- | Fribidi | Unicode document |
- |---------------+--------------------------|
- | | overflow isolate count |
- | ~over_pushed~ | overflow embedding count |
- | ~isolate~ | valid isolate count |
-* <2015-05-29 Fri>
-** Comparing with the bidi ref test
-The following scenario was used for comparing with the bidi reference
- - Download and compile the bidiref test with the following SConstruct file in file:../BidiReferenceC/6.3.0/source
-#+BEGIN_SRC python
-import glob
-
-env = Environment(CPPFLAGS=['-Wall','-g'],
- CPPPATH=['../include'])
-
-env.Program('bidiref',
- glob.glob('*.c'))
-#+END_SRC
- - Run test suite on fribidi with:
-#+BEGIN_SRC sh
-cd ../fribidi-vs-unicode
-./test BidiTest.txt | head -20
-#+END_SRC
- - Create a file ~BidiTestSmall.txt~ that only contain the failing test. E.g.:
-#+BEGIN_EXAMPLE
-@Levels: 0 x x 1
-@Reorder: 0 3
-RLI LRE PDF R; 3
-#+END_EXAMPLE
- - Run the bidiref test on this single test with:
-#+BEGIN_SRC sh
-./bidiref -d4 -x BidiTestSmall.txt
-#+END_SRC
- - Note that bidiref contains magic to recognize the unicode file format only if the input file name contains the string "BidiTest".
- - An example failing run for fribidi is:
-#+BEGIN_EXAMPLE
-> ./test b.txt
-failure on line 4
-input is: RLI LRE PDF R; 3
-base dir: auto
-expected levels: 0 x x 1
-returned levels: 0 0 0 3
-expected order: 0 3
-returned order: 0 3
-#+END_EXAMPLE
- - Goal is to debug, compare, and fix all ~BidiTestSmall.txt~ until the entire file passes.
-** Errors
- - [X] Line 63038: RLI LRE PDF R; 3
- - [ ] Line 66856: R RLI R .
-*** R RLI R
- - Expected levels vs fribidi levels: 1 0 1 vs 1 1 1
- - Don't understand the difference between the embedding level and the paragraph embedding level.
- - How can a level be lower than the paragraph level?
-* <2015-05-31 Sun>
-** Regarding R RLI R
- - The problem is that N1 and N2 are done over isolating boundaries. This needs to be fixed! Either by remembering the isolating sequence level of each run and make sure to connect only runs of the same level, or alternatively to always sweep and count. Both of these solutions are slow as they require sweeping over inner runs. A faster solution would be to have a pointer to the next run of the same sequence. Can this be done efficiently?
-* <2015-06-05 Fri>
- - [ ] make macros ~PREV_TYPE_SKIP_ISOLATE~ and ~NEXT_TYPE_SKIP_ISOLATE~ macros based on the code in N1.
-* <2015-06-18 Thu>
-** NSI merging problems
- - The following ~b.txt~ shows that there is a difference between an isolated NSM and one preceded by an isolating sequence.
-#+BEGIN_EXAMPLE
-@Levels: 1
-@Reorder: 0
-NSM; 4
-
-@Levels: 1 1 1
-@Reorder: 2 1 0
-LRI PDI NSM; 4
-#+END_EXAMPLE
- - In both these cases, because of the RTL direction, the NSM should get the type of the base direction. This currently does not happen. See TBDov
-* <2015-06-20 Sat>
-** Test progress
- | Date | Num fail | First line fail | First fail |
- |----------------+----------+-----------------+--------------------------------------------|
- | 2015-06-20 Sat | 139 | 236713 | |
- | | 22 | 497052 | R ON FSI L PDI LRI L PDI RLI L PDI ON R; 2 |
-** Thoughts
- - The quest of finding a single strategy for finding the next and previous types seem to fail. It seems like every rule is looking for something different.
- - The real problem is that sequences like ~R OL LRI ... PDI OL R~ should really be interpreted as ~R OL OL OL R~ for the algorithm to work. But in contrast to ~R OL OL OL~ that is compacted as ~R OL×3 R~ and which may be matched by a simple forward and backward match, in ~LRI ... PDI~ the right hand R must be found by scanning. This is what Ι tried doing in my changes to fribidi so far, but it failed due to differen needs.
- - My strategy is first finding a scanning strategy that works, and only afterwards optimizing it.
-* <2015-06-21 Sun>
-** Test progress
- | Date | Num fail | First line fail | First fail |
- |----------------+----------+-----------------+-------------------------------------------------|
- | 2015-06-21 Sat | 18 | 236713 | AL ON FSI L PDI LRI L PDI RLI R PDI ON ET EN; 2 |
-
- - Note: I believe this is a bug...
-* <2015-08-23 Sun>
-** Misc
- - All the unicode files should be moved to a common place, as ~BidiTest.txt~ and ~BidiCharacterTest.txt~ are needed for testing. This means moving out the ~UnicodeData.txt~ from ~gen.tab~ e.g. to ~.../fribid/Unidata~.
- - Create a new branch IsoLinks that adds FriBidiRun, ~next_iso~ and ~prev_iso~ for next and previous isolate runs.
-f - ~test.c~ should probably be renamed to ~test-bidi-test-txt.c~ and ~test-character~ should be changed to ~test-bidi-character-test-txt.c~ to reflect what they test.
-* <2016-01-24 Sun>
-** Packing stuff
- - [ ] Should we make sure that bootstrap and dependencies is clean? Currently lots of warnings on my system:
-#+BEGIN_SRC sh
-configure.ac:201: warning: LT_INIT was called before AM_PROG_AR
-/usr/share/aclocal-1.15/ar-lib.m4:13: AM_PROG_AR is expanded from...
-configure.ac:201: the top level
-configure.ac:201: warning: LT_INIT was called before AM_PROG_AR
-aclocal.m4:9317: AM_PROG_AR is expanded from...
-configure.ac:201: the top level
-libtoolize: putting auxiliary files in '.'.
-libtoolize: copying file './ltmain.sh'
-libtoolize: Consider adding 'AC_CONFIG_MACRO_DIRS([m4])' to configure.ac,
-libtoolize: and rerunning libtoolize and aclocal.
-libtoolize: Consider adding '-I m4' to ACLOCAL_AMFLAGS in Makefile.am.
-configure.ac:201: warning: LT_INIT was called before AM_PROG_AR
-/usr/share/aclocal-1.15/ar-lib.m4:13: AM_PROG_AR is expanded from...
-configure.ac:201: the top level
-configure.ac:201: warning: LT_INIT was called before AM_PROG_AR
-aclocal.m4:9317: AM_PROG_AR is expanded from...
-configure.ac:201: the top level
-configure.ac:201: warning: LT_INIT was called before AM_PROG_AR
-aclocal.m4:9317: AM_PROG_AR is expanded from...
-configure.ac:201: the top level
-configure.ac:201: warning: LT_INIT was called before AM_PROG_AR
-aclocal.m4:9317: AM_PROG_AR is expanded from...
-configure.ac:201: the top level
-configure.ac:55: installing './compile'
-configure.ac:50: installing './missing'
-bin/Makefile.am: installing './depcomp'
-lib/Headers.mk:22: warning: shell cat $(top_srcdir: non-POSIX variable name
-lib/Headers.mk:22: (probably a GNU make extension)
-doc/Makefile.am:26: 'lib/Headers.mk' included from here
-lib/Headers.mk:22: warning: shell cat $(top_srcdir: non-POSIX variable name
-lib/Headers.mk:22: (probably a GNU make extension)
-lib/Makefile.am:28: 'lib/Headers.mk' included from here
-test/Makefile.am:30: warning: '%'-style pattern rules are a GNU make extension
-#+END_SRC
- - configure ok.
- - make passes but outputs warnings about missing c2man.
-#+BEGIN_SRC sh
-WARNING: 'c2man' is missing on your system.
- You might have modified some files without having the proper
- tools for further handling them. Check the 'README' file, it
- often tells you about the needed prerequisites for installing
- this package. You may also peek at any GNU archive site, in
- case some other package contains this missing 'c2man' program.
-#+END_SRC
- - Found c2man at http://www.ciselant.de/c2man/c2man.html , but it is not compatible with the make file usage.
- - "make test" doesn't do anything.
-* <2016-01-25 Mon>
-** Release engineering
- - Got "new" c2man from Ubuntu archive at https://answers.launchpad.net/ubuntu/+source/c2man/2.41-18
- - Compiled as follows:
-#+BEGIN_SRC sh
-tar -xf ~/hd/Download/c2man_2.41.orig.tar.gz
-mv c2man-2.41.orig c2man-2.41-18
-cd c2man-2.41-18
-zcat ~/hd/Download/c2man_2.41-18.diff.gz| patch -p1
-./Configure
-make install
-#+END_SRC
- - Verified that man pages are reasonoble (after fix to invocation of c2man).
- - [ ] Wait for response from Behdad about the issues in mail I sent.
- - [X] Rename unicode63 branch to dov-unicode63 branch.
- - [X] Create new branch unicode63 out of master.
- - [X] Manually cherry pick from dov-unicode63 changes.
- - [X] Squash and rebase dov-unicode63 on top of unicode63.
- - [ ] Pull request to Behdad (after got mail)
-** Testing
-#+BEGIN_SRC sh
-env LD_LIBRARY_PATH=../lib/.libs ./test BidiTest.txt
-#+END_SRC
- - Still 15 errors.
-* <2016-01-28 Thu>
-** Investigating failure
- - Not clear to me the correctness of the following test:
-#+BEGIN_EXAMPLE
-@Levels: 0 0 x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x 124 124 124
-@Reorder: 0 1 66 67 68
-L WS LRO LRO LRO LRO LRO LRO LRO LRO LRO LRO LRO LRO LRO LRO LRO LRO LRO LRO LRO LRO LRO LRO LRO LRO LRO LRO LRO LRO LRO LRO LRO LRO LRO LRO LRO LRO LRO LRO LRO LRO LRO LRO LRO LRO LRO LRO LRO LRO LRO LRO LRO LRO LRO LRO LRO LRO LRO LRO LRO LRO LRO LRO LRO RLO L L L; 2
-#+END_EXAMPLE
- - Why doesn't the RLO raise the level to 125?
- - Compare with Behdads file:../pybyedie . Oops! Doesn't support 6.3. :-(
- - Found reason! There is one more LRO than Ι thought that wants to push the level to 126 which is illegal. We therefore get overflow which should prevent the RLO from increasing the level further!
- - [ ] Figure out why this does not happen.
-* <2016-01-29 Fri>
-** Failure
- - Added prevention of going from 124 → 125 in case of overflow. Solved 3 errors.
- - [ ] Solve 12 remaining errors. :-)
-* <2016-01-30 Sat>
-** Compiling with debug
-#+BEGIN_SRC sh
-env CFLAGS='-fPIC -g -O0' ./configure --prefix=/usr/local --with-pic
-#+END_SRC
-** Failures
- - Next failure:
-#+BEGIN_EXAMPLE
-@Levels: x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x 124 x 125 x 125 x x x x 125 x 125 x 124
-@Reorder: 62 73 71 66 64 75
-LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE ON RLO L LRE RLI LRE RLE LRO RLO PDI PDF L PDF ON; 1
-#+END_EXAMPLE
- - The resulting discrepancy is:
-#+BEGIN_EXAMPLE
- Exp Ret
-:
-61: LRE x 0
-62: ON 124 124
-63: RLO x 124
-64: L 125 125
-65: LRE x 125
-66: RLI 125 125
-67: LRE x 125
-68: RLE x 125
-69: LRO x 125
-70: RLO x 125
-71: PDI 125 1
-72: PDF x 1
-73: L 125 125
-74: PDF x 125
-75: ON; 124 124
-#+END_EXAMPLE
- - Solved by
-#+BEGIN_EXAMPLE
-@@ -539,7 +543,11 @@ fribidi_get_par_embedding_levels (
- for (i = RL_LEN (pp); i; i--)
- {
- if (isolate_overflow > 0)
-- isolate_overflow--;
-+ {
-+ isolate_overflow--;
-+ RL_LEVEL (pp) = level;
-+ }
-#+END_EXAMPLE
- - Next bug.
-#+BEGIN_EXAMPLE
-@Levels: x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x 123 x 124 124 124 x x x x 124 124 124 x 123
-@Reorder: 75 64 65 66 71 72 73 62
-LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE RLE ON LRO R LRI R LRE RLE LRO RLO R PDI R PDF ON; 1
-#+END_EXAMPLE
- - which yielded:
-#+BEGIN_EXAMPLE
-62: ON 123 123
-63: LRO x 123
-64: R 124 124
-65: LRI 124 124
-66: R 124 124
-67: LRE x 124
-68: RLE x 124
-69: LRO x 124
-70: RLO x 124
-71: R 124 125 (x)
-72: PDI 124 125 (x)
-73: R 124 125 (x)
-74: PDF x 125
-75: ON 123 124 (x)
-#+END_EXAMPLE
- - The problem was that character 71 was increased even though we were in isolate mode.
- - Fixed!
- - Next error:
-#+BEGIN_EXAMPLE
-failure on line 497576
-input is: LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE RLE ON LRO R RLI ON LRO RLE RLO LRE ON PDI R PDF ON; 7
-base dir: auto
-expected levels: x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x 123 x 124 124 125 x x x x 125 124 124 x 123
-returned levels: 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 123 123 124 124 125 125 125 125 125 125 125 125 125 125
-expected order: 75 64 65 71 66 72 73 62
-returned order: 64 65 75 73 72 71 66 62
-#+END_EXAMPLE
- - with output:
-#+BEGIN_EXAMPLE
-60: LRE x 1
-61: RLE x 1
-62: ON 123 123
-63: LRO x 123
-64: R 124 124
-65: RLI 124 124
-66: ON 125 125
-67: LRO x 125 overflow++
-68: RLE x 125 overflow++
-69: RLO x 125 overflow++
-70: LRE x 125 overflow++
-71: ON 125 125
-72: PDI 124 125 (x) should pop to RLI@65! -> 124
-73: R 124 125 (x)
-74: PDF x 125 should pop to LRO@63 -> 123
-75: ON; 123 125 (x)
-#+END_EXAMPLE
- - The problem was that the PDI did not reset the overpushed level! Once it was inserted all tests pass.
- - Patch that solved the problems:
-#+BEGIN_EXAMPLE
-@@ -281,6 +281,7 @@ print_bidi_string (
- #define PUSH_STATUS \
- FRIBIDI_BEGIN_STMT \
- if LIKELY(over_pushed == 0 \
-+ && isolate_overflow == 0 \
- && new_level <= FRIBIDI_BIDI_MAX_EXPLICIT_LEVEL) \
- { \
- if UNLIKELY(level == FRIBIDI_BIDI_MAX_EXPLICIT_LEVEL - 1) \
-@@ -551,6 +552,7 @@ fribidi_get_par_embedding_levels (
- terminated by the PDI */
- while (stack_size && !status_stack[stack_size-1].isolate)
- POP_STATUS;
-+ over_pushed = 0; /* The PDI resets the overpushed! */
- POP_STATUS;
- isolate_level-- ;
- valid_isolate_count--;
-#+END_EXAMPLE