diff options
345 files changed, 13830 insertions, 4542 deletions
diff --git a/.dir-locals.el b/.dir-locals.el index 970aecb551..09d02274c3 100644 --- a/.dir-locals.el +++ b/.dir-locals.el @@ -5,11 +5,9 @@ (java-mode (indent-tabs-mode . nil)) (perl-mode (indent-tabs-mode . nil)) (xml-mode (indent-tabs-mode . nil)) - ;; In C code indentation is 4 spaces and in C++ 2 spaces - ;; erts/ overrides C++ to 4 spaces (c++-mode (indent-tabs-mode . nil) - (c-basic-offset . 2)) + (c-basic-offset . 4)) (c-mode (indent-tabs-mode . nil) (c-basic-offset . 4))) diff --git a/.editorconfig b/.editorconfig new file mode 100644 index 0000000000..e9d6ce9c89 --- /dev/null +++ b/.editorconfig @@ -0,0 +1,22 @@ +; This file is for unifying the coding style for different editors and IDEs. +; More information at http://EditorConfig.org +; +; To use this from emacs install the editorconfig package + +root = true + +[*] +end_of_line = LF + +[*] +indent_style = space +indent_size = 4 +tab_width = 8 +insert_final_newline = true +# Adding trim_trailing_whitespace removes trailing +# from entire buffer on save and we do not want that +# trim_trailing_whitespace = true + +[{Makefile*,**.mk*}] +# Use tabs for indentation (Makefiles require tabs) +indent_style = tab diff --git a/.github/scripts/init-pre-release.sh b/.github/scripts/init-pre-release.sh index 7bc0550b53..7a86f3052c 100755 --- a/.github/scripts/init-pre-release.sh +++ b/.github/scripts/init-pre-release.sh @@ -1,26 +1,6 @@ #!/bin/sh -## We create a tar ball that has generated configure -## (old versions might not have these included) -## This is used later by build-otp-tar to create -## the pre-built tar ball +## We create a tar ball that is used later by build-otp-tar +## to create the pre-built tar ball -if [ -f ./configure ]; then - git archive --prefix otp/ -o otp_src.tar.gz HEAD -else - ERL_TOP=`pwd` - ./otp_build autoconf - find . -name aclocal.m4 | xargs git add -f - find . -name configure | xargs git add -f - find . -name config.h.in | xargs git add -f - find . -name config.guess | xargs git add -f - find . -name config.sub | xargs git add -f - find . -name install-sh | xargs git add -f - if ! git config user.name; then - git config user.email "you@example.com" - git config user.name "Your Name" - fi - git commit --no-verify -m 'Add generated configure files' - git archive --prefix otp/ -o otp_src.tar.gz HEAD - git reset --hard HEAD~1 -fi +git archive --prefix otp/ -o otp_src.tar.gz HEAD diff --git a/.github/workflows/main.yaml b/.github/workflows/main.yaml index 1d1f2b44bc..d22a16647e 100644 --- a/.github/workflows/main.yaml +++ b/.github/workflows/main.yaml @@ -282,3 +282,8 @@ jobs: artifacts/*.txt env: GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} + - name: Deploy on erlang.org + env: + GITHUB_TOKEN: ${{ secrets.TRIGGER_ERLANG_ORG_BUILD }} + run: | + curl -H "Authorization: token ${GITHUB_TOKEN}" -X POST -H "Accept: application/vnd.github.v3+json" "https://api.github.com/repos/erlang/erlang-org/actions/workflows/update-gh-cache.yaml/dispatches" -d '{"ref":"master"}' diff --git a/HOWTO/OTP-PATCH-APPLY.md b/HOWTO/OTP-PATCH-APPLY.md index 1fdd13f9c4..2340784992 100644 --- a/HOWTO/OTP-PATCH-APPLY.md +++ b/HOWTO/OTP-PATCH-APPLY.md @@ -49,11 +49,6 @@ the updated applications. > *NOTE*: Before applying a patch you need to do a *full* build > of OTP in the source directory. -If you are building in `git` you first need to generate the -`configure` scripts: - - $ ./otp_build autoconf - Configure and build all applications in OTP: $ configure diff --git a/HOWTO/TESTING.md b/HOWTO/TESTING.md index 7a7f6982f2..f713f85231 100644 --- a/HOWTO/TESTING.md +++ b/HOWTO/TESTING.md @@ -139,7 +139,7 @@ purpose. The `make test` command works when the current directory contains a directory called test and in the root directory of the source code tree. -*(Waring)* Some test cases do not run correctly or cannot be run at +*(Warning)* Some test cases do not run correctly or cannot be run at all through the `make test` command (typically test cases that require test specific C code to be compiled) because `make test` runs tests directly by invoking the `ct_run` command instead of using the `ts` diff --git a/OTP_VERSION b/OTP_VERSION index beafbdd5e8..dcc868f355 100644 --- a/OTP_VERSION +++ b/OTP_VERSION @@ -1 +1 @@ -24.1.2 +24.1.7 @@ -41,7 +41,17 @@ To compile Erlang from source, run the following commands. The complete building ``` git clone https://github.com/erlang/otp.git cd otp +``` +Checkout the branch or tag of your choice +``` +git checkout maint-24 # current latest stable version +``` +For older versions run autoconf +``` ./otp_build autoconf +``` +Configure, build and install +``` ./configure make make install diff --git a/bootstrap/bin/no_dot_erlang.boot b/bootstrap/bin/no_dot_erlang.boot Binary files differindex 3e0d3f3601..76f70a4e1a 100644 --- a/bootstrap/bin/no_dot_erlang.boot +++ b/bootstrap/bin/no_dot_erlang.boot diff --git a/bootstrap/bin/start.boot b/bootstrap/bin/start.boot Binary files differindex 3e0d3f3601..76f70a4e1a 100644 --- a/bootstrap/bin/start.boot +++ b/bootstrap/bin/start.boot diff --git a/bootstrap/bin/start_clean.boot b/bootstrap/bin/start_clean.boot Binary files differindex 3e0d3f3601..76f70a4e1a 100644 --- a/bootstrap/bin/start_clean.boot +++ b/bootstrap/bin/start_clean.boot diff --git a/bootstrap/lib/compiler/ebin/beam_asm.beam b/bootstrap/lib/compiler/ebin/beam_asm.beam Binary files differindex f1bccf56dc..559a6df9c3 100644 --- a/bootstrap/lib/compiler/ebin/beam_asm.beam +++ b/bootstrap/lib/compiler/ebin/beam_asm.beam diff --git a/bootstrap/lib/kernel/ebin/gen_udp_socket.beam b/bootstrap/lib/kernel/ebin/gen_udp_socket.beam Binary files differindex 9bc07a711e..e2bd99efb1 100644 --- a/bootstrap/lib/kernel/ebin/gen_udp_socket.beam +++ b/bootstrap/lib/kernel/ebin/gen_udp_socket.beam diff --git a/bootstrap/lib/kernel/ebin/inet_dns.beam b/bootstrap/lib/kernel/ebin/inet_dns.beam Binary files differindex 2f27439f9b..dd8ec792ba 100644 --- a/bootstrap/lib/kernel/ebin/inet_dns.beam +++ b/bootstrap/lib/kernel/ebin/inet_dns.beam diff --git a/bootstrap/lib/kernel/ebin/logger.beam b/bootstrap/lib/kernel/ebin/logger.beam Binary files differindex 7015f8dbf7..05774930eb 100644 --- a/bootstrap/lib/kernel/ebin/logger.beam +++ b/bootstrap/lib/kernel/ebin/logger.beam diff --git a/bootstrap/lib/stdlib/ebin/erl_error.beam b/bootstrap/lib/stdlib/ebin/erl_error.beam Binary files differindex 6538b9170b..3a0a7d1110 100644 --- a/bootstrap/lib/stdlib/ebin/erl_error.beam +++ b/bootstrap/lib/stdlib/ebin/erl_error.beam diff --git a/bootstrap/lib/stdlib/ebin/erl_lint.beam b/bootstrap/lib/stdlib/ebin/erl_lint.beam Binary files differindex 3d1c5c77a4..225ee318c2 100644 --- a/bootstrap/lib/stdlib/ebin/erl_lint.beam +++ b/bootstrap/lib/stdlib/ebin/erl_lint.beam diff --git a/bootstrap/lib/stdlib/ebin/erl_stdlib_errors.beam b/bootstrap/lib/stdlib/ebin/erl_stdlib_errors.beam Binary files differindex 742bd0d79c..84a8af57c7 100644 --- a/bootstrap/lib/stdlib/ebin/erl_stdlib_errors.beam +++ b/bootstrap/lib/stdlib/ebin/erl_stdlib_errors.beam diff --git a/bootstrap/lib/stdlib/ebin/shell_docs.beam b/bootstrap/lib/stdlib/ebin/shell_docs.beam Binary files differindex 04d5c41d9f..edd19f9c8d 100644 --- a/bootstrap/lib/stdlib/ebin/shell_docs.beam +++ b/bootstrap/lib/stdlib/ebin/shell_docs.beam diff --git a/bootstrap/lib/stdlib/ebin/supervisor.beam b/bootstrap/lib/stdlib/ebin/supervisor.beam Binary files differindex 8f7186146e..3f9c465368 100644 --- a/bootstrap/lib/stdlib/ebin/supervisor.beam +++ b/bootstrap/lib/stdlib/ebin/supervisor.beam diff --git a/erts/.dir-locals.el b/erts/.dir-locals.el deleted file mode 100644 index 37b636dfad..0000000000 --- a/erts/.dir-locals.el +++ /dev/null @@ -1,6 +0,0 @@ -;; erts Emacs settings -( - ;; In C++ code indentation is 4 spaces - (c++-mode - (indent-tabs-mode . nil) - (c-basic-offset . 4))) diff --git a/erts/.gitignore b/erts/.gitignore index cc6c270df4..8886a8855e 100644 --- a/erts/.gitignore +++ b/erts/.gitignore @@ -10,6 +10,7 @@ /etc/common/Install /etc/common/erl.src /etc/unix/etp-commands +/etc/win32/version.h /test/Emakefile /test/*.beam @@ -20,3 +21,4 @@ /emulator/pcre/pcre_exec_loop_break_cases.inc /emulator/beam/erl_db_insert_list.ycf.h + diff --git a/erts/config.h.in b/erts/config.h.in index afdb89d5cc..8eb0d28bf9 100644 --- a/erts/config.h.in +++ b/erts/config.h.in @@ -83,6 +83,18 @@ /* ESOCK counter size */ #undef ESOCK_COUNTER_SIZE +/* Interface hwaddr supported */ +#undef ESOCK_USE_HWADDR + +/* Interface ifindex supported */ +#undef ESOCK_USE_IFINDEX + +/* Interface map supported */ +#undef ESOCK_USE_IFMAP + +/* Interface index supported */ +#undef ESOCK_USE_INDEX + /* Use SO_[RCV|SND]TMIEO */ #undef ESOCK_USE_RCVSNDTIMEO @@ -983,6 +995,15 @@ /* Define to 1 if `ifr_hwaddr' is a member of `struct ifreq'. */ #undef HAVE_STRUCT_IFREQ_IFR_HWADDR +/* Define to 1 if `ifr_ifindex' is a member of `struct ifreq'. */ +#undef HAVE_STRUCT_IFREQ_IFR_IFINDEX + +/* Define to 1 if `ifr_index' is a member of `struct ifreq'. */ +#undef HAVE_STRUCT_IFREQ_IFR_INDEX + +/* Define to 1 if `ifr_map' is a member of `struct ifreq'. */ +#undef HAVE_STRUCT_IFREQ_IFR_MAP + /* Define to 1 if `assoc_id' is a member of `struct sctp_accoc_value'. */ #undef HAVE_STRUCT_SCTP_ACCOC_VALUE_ASSOC_ID diff --git a/erts/configure b/erts/configure index 85a3e20a05..8b6aad3f1c 100755 --- a/erts/configure +++ b/erts/configure @@ -773,7 +773,6 @@ infodir docdir oldincludedir includedir -runstatedir localstatedir sharedstatedir sysconfdir @@ -920,7 +919,6 @@ datadir='${datarootdir}' sysconfdir='${prefix}/etc' sharedstatedir='${prefix}/com' localstatedir='${prefix}/var' -runstatedir='${localstatedir}/run' includedir='${prefix}/include' oldincludedir='/usr/include' docdir='${datarootdir}/doc/${PACKAGE}' @@ -1173,15 +1171,6 @@ do | -silent | --silent | --silen | --sile | --sil) silent=yes ;; - -runstatedir | --runstatedir | --runstatedi | --runstated \ - | --runstate | --runstat | --runsta | --runst | --runs \ - | --run | --ru | --r) - ac_prev=runstatedir ;; - -runstatedir=* | --runstatedir=* | --runstatedi=* | --runstated=* \ - | --runstate=* | --runstat=* | --runsta=* | --runst=* | --runs=* \ - | --run=* | --ru=* | --r=*) - runstatedir=$ac_optarg ;; - -sbindir | --sbindir | --sbindi | --sbind | --sbin | --sbi | --sb) ac_prev=sbindir ;; -sbindir=* | --sbindir=* | --sbindi=* | --sbind=* | --sbin=* \ @@ -1319,7 +1308,7 @@ fi for ac_var in exec_prefix prefix bindir sbindir libexecdir datarootdir \ datadir sysconfdir sharedstatedir localstatedir includedir \ oldincludedir docdir infodir htmldir dvidir pdfdir psdir \ - libdir localedir mandir runstatedir + libdir localedir mandir do eval ac_val=\$$ac_var # Remove trailing slashes. @@ -1472,7 +1461,6 @@ Fine tuning of the installation directories: --sysconfdir=DIR read-only single-machine data [PREFIX/etc] --sharedstatedir=DIR modifiable architecture-independent data [PREFIX/com] --localstatedir=DIR modifiable single-machine data [PREFIX/var] - --runstatedir=DIR modifiable per-process data [LOCALSTATEDIR/run] --libdir=DIR object code libraries [EPREFIX/lib] --includedir=DIR C header files [PREFIX/include] --oldincludedir=DIR C header files for non-gcc [/usr/include] @@ -14102,6 +14090,78 @@ $as_echo "#define ESOCK_USE_SOCKET_REGISTRY 1" >>confdefs.h fi +ac_fn_c_check_member "$LINENO" "struct ifreq" "ifr_map" "ac_cv_member_struct_ifreq_ifr_map" "#ifdef __WIN32__ + #else + #include <net/if.h> + #endif + +" +if test "x$ac_cv_member_struct_ifreq_ifr_map" = xyes; then : + +cat >>confdefs.h <<_ACEOF +#define HAVE_STRUCT_IFREQ_IFR_MAP 1 +_ACEOF + + +$as_echo "#define ESOCK_USE_IFMAP /**/" >>confdefs.h + +fi + + +ac_fn_c_check_member "$LINENO" "struct ifreq" "ifr_hwaddr" "ac_cv_member_struct_ifreq_ifr_hwaddr" "#ifdef __WIN32__ + #else + #include <net/if.h> + #endif + +" +if test "x$ac_cv_member_struct_ifreq_ifr_hwaddr" = xyes; then : + +cat >>confdefs.h <<_ACEOF +#define HAVE_STRUCT_IFREQ_IFR_HWADDR 1 +_ACEOF + + +$as_echo "#define ESOCK_USE_HWADDR /**/" >>confdefs.h + +fi + + +ac_fn_c_check_member "$LINENO" "struct ifreq" "ifr_ifindex" "ac_cv_member_struct_ifreq_ifr_ifindex" "#ifdef __WIN32__ + #else + #include <net/if.h> + #endif + +" +if test "x$ac_cv_member_struct_ifreq_ifr_ifindex" = xyes; then : + +cat >>confdefs.h <<_ACEOF +#define HAVE_STRUCT_IFREQ_IFR_IFINDEX 1 +_ACEOF + + +$as_echo "#define ESOCK_USE_IFINDEX /**/" >>confdefs.h + +fi + + +ac_fn_c_check_member "$LINENO" "struct ifreq" "ifr_index" "ac_cv_member_struct_ifreq_ifr_index" "#ifdef __WIN32__ + #else + #include <net/if.h> + #endif + +" +if test "x$ac_cv_member_struct_ifreq_ifr_index" = xyes; then : + +cat >>confdefs.h <<_ACEOF +#define HAVE_STRUCT_IFREQ_IFR_INDEX 1 +_ACEOF + + +$as_echo "#define ESOCK_USE_INDEX /**/" >>confdefs.h + +fi + + #-------------------------------------------------------------------- # Check for the existence of the -lsocket and -lnsl libraries. diff --git a/erts/configure.in b/erts/configure.in index d8ad6ff552..d7828ca14f 100644 --- a/erts/configure.in +++ b/erts/configure.in @@ -1475,6 +1475,42 @@ AS_IF([test "x$enable_esock_socket_registry" = "xno"], [ [Use socket registry by default])]) +AC_CHECK_MEMBERS([struct ifreq.ifr_map], + [AC_DEFINE(ESOCK_USE_IFMAP, [], [Interface map supported])], + [], + [#ifdef __WIN32__ + #else + #include <net/if.h> + #endif + ]) + +AC_CHECK_MEMBERS([struct ifreq.ifr_hwaddr], + [AC_DEFINE(ESOCK_USE_HWADDR, [], [Interface hwaddr supported])], + [], + [#ifdef __WIN32__ + #else + #include <net/if.h> + #endif + ]) + +AC_CHECK_MEMBERS([struct ifreq.ifr_ifindex], + [AC_DEFINE(ESOCK_USE_IFINDEX, [], [Interface ifindex supported])], + [], + [#ifdef __WIN32__ + #else + #include <net/if.h> + #endif + ]) + +AC_CHECK_MEMBERS([struct ifreq.ifr_index], + [AC_DEFINE(ESOCK_USE_INDEX, [], [Interface index supported])], + [], + [#ifdef __WIN32__ + #else + #include <net/if.h> + #endif + ]) + dnl dnl This test kindly borrowed from Tcl diff --git a/erts/doc/src/erl_ext_dist.xml b/erts/doc/src/erl_ext_dist.xml index 06e9ac639d..a4fa3449cf 100644 --- a/erts/doc/src/erl_ext_dist.xml +++ b/erts/doc/src/erl_ext_dist.xml @@ -117,23 +117,34 @@ <cell align="center"><c>Data</c></cell> </row> <tcaption>Compressed Data Format when Expanded</tcaption></table> + </section> + + <section> <marker id="utf8_atoms"/> - <note> - <p>As from ERTS 9.0 (OTP 20), atoms may contain any Unicode - characters and are always encoded using the UTF-8 external formats - <seeguide marker="#ATOM_UTF8_EXT"><c>ATOM_UTF8_EXT</c></seeguide> - or <seeguide marker="#SMALL_ATOM_UTF8_EXT"><c>SMALL_ATOM_UTF8_EXT</c></seeguide>. - The old Latin-1 formats <seeguide marker="#ATOM_EXT"><c>ATOM_EXT</c></seeguide> - and <seeguide marker="#SMALL_ATOM_EXT"><c>SMALL_ATOM_EXT</c></seeguide> - are deprecated and are only kept for backward - compatibility when decoding terms encoded by older nodes.</p> - <p>Support for UTF-8 encoded atoms in the external format has been - available since ERTS 5.10 (OTP R16). This ability allows such old nodes - to decode, store and encode any Unicode atoms received from a new OTP 20 - node.</p> - <p>The maximum number of allowed characters in an atom is 255. In the - UTF-8 case, each character can need 4 bytes to be encoded.</p> - </note> + <title>Encoding atoms</title> + <p> + As from ERTS 9.0 (OTP 20), atoms may contain any Unicode characters. + </p> + <p> + Atoms sent over node distribution are always encoded in UTF-8 using + either <seeguide marker="#ATOM_UTF8_EXT"><c>ATOM_UTF8_EXT</c></seeguide>, + <seeguide marker="#SMALL_ATOM_UTF8_EXT"><c>SMALL_ATOM_UTF8_EXT</c></seeguide> + or <seeguide marker="#ATOM_CACHE_REF"><c>ATOM_CACHE_REF</c></seeguide>. + </p> + <p> + Atoms encoded with <seemfa marker="erts:erlang#term_to_binary/1"> + <c>erlang:term_to_binary/1,2</c></seemfa> or + <seemfa marker="erts:erlang#term_to_iovec/1"> + <c>erlang:term_to_iovec/1,2</c></seemfa> are by default still using the + old deprecated Latin-1 format + <seeguide marker="#ATOM_EXT"><c>ATOM_EXT</c></seeguide> + for atoms that only contain Latin-1 characters (Unicode code points + 0-255). Atoms with higher code points will be encoded in UTF-8 using + either <seeguide marker="#ATOM_UTF8_EXT"><c>ATOM_UTF8_EXT</c></seeguide> or + <seeguide marker="#SMALL_ATOM_UTF8_EXT"><c>SMALL_ATOM_UTF8_EXT</c></seeguide>. + </p> + <p>The maximum number of allowed characters in an atom is 255. In the + UTF-8 case, each character can need 4 bytes to be encoded.</p> </section> <section> @@ -283,8 +294,8 @@ </p> <p> For more information on encoding of atoms, see the - <seeguide marker="#utf8_atoms">note on UTF-8 encoded atoms</seeguide> - in the beginning of this section. + <seeguide marker="#utf8_atoms">section on UTF-8 encoded atoms</seeguide> + above. </p> <p> If the <c>NewCacheEntryFlag</c> for the next <c>AtomCacheRef</c> @@ -666,10 +677,8 @@ <p> Encodes a port identifier (obtained from <seemfa marker="erlang#open_port/2"><c>erlang:open_port/2</c></seemfa>). - <c>Node</c> is an encoded atom, that is, - <seeguide marker="#ATOM_UTF8_EXT"><c>ATOM_UTF8_EXT</c></seeguide>, - <seeguide marker="#SMALL_ATOM_UTF8_EXT"><c>SMALL_ATOM_UTF8_EXT</c></seeguide> - or <seeguide marker="#ATOM_CACHE_REF"><c>ATOM_CACHE_REF</c></seeguide>. + <c>Node</c> is the originating node, + <seeguide marker="#utf8_atoms">encoded as an atom</seeguide>. <c>ID</c> is a 64-bit big endian unsigned integer. The <c>Creation</c> works just like in <seeguide marker="#NEW_PID_EXT"><c>NEW_PID_EXT</c></seeguide>. Port operations are not allowed across node boundaries. @@ -729,11 +738,8 @@ </p> <taglist> <tag><c>Node</c></tag> - <item><p>The name of the originating node, encoded using - <seeguide marker="#ATOM_UTF8_EXT"><c>ATOM_UTF8_EXT</c></seeguide>, - <seeguide marker="#SMALL_ATOM_UTF8_EXT"><c>SMALL_ATOM_UTF8_EXT</c></seeguide> - or <seeguide - marker="#ATOM_CACHE_REF"><c>ATOM_CACHE_REF</c></seeguide>.</p> + <item><p>The name of the originating node, + <seeguide marker="#utf8_atoms">encoded as an atom</seeguide>.</p> </item> <tag><c>ID</c></tag> <item><p>A 32-bit big endian unsigned integer. If distribution flag @@ -1072,10 +1078,8 @@ </p> <taglist> <tag><c>Node</c></tag> - <item><p>The name of the originating node, encoded using - <seeguide marker="#ATOM_UTF8_EXT"><c>ATOM_UTF8_EXT</c></seeguide>, - <seeguide marker="#SMALL_ATOM_UTF8_EXT"><c>SMALL_ATOM_UTF8_EXT</c></seeguide> - or <seeguide marker="#ATOM_CACHE_REF"><c>ATOM_CACHE_REF</c></seeguide>.</p> + <item><p>The name of the originating node, + <seeguide marker="#utf8_atoms">encoded as an atom</seeguide>.</p> </item> <tag><c>Len</c></tag> <item><p>A 16-bit big endian unsigned integer not larger than 5 when the @@ -1137,12 +1141,9 @@ </item> <tag><c>Module</c></tag> <item> - <p>Encoded as an atom, using - <seeguide marker="#ATOM_UTF8_EXT"><c>ATOM_UTF8_EXT</c></seeguide>, - <seeguide marker="#SMALL_ATOM_UTF8_EXT"><c>SMALL_ATOM_UTF8_EXT</c></seeguide>, - or <seeguide marker="#ATOM_CACHE_REF"> - <c>ATOM_CACHE_REF</c></seeguide>. - This is the module that the fun is implemented in. + <p> + The module that the fun is implemented in, + <seeguide marker="#utf8_atoms">encoded as an atom</seeguide>. </p> </item> <tag><c>Index</c></tag> @@ -1232,13 +1233,10 @@ </item> <tag><c>Module</c></tag> <item> - <p>Encoded as an atom, using - <seeguide marker="#ATOM_EXT"><c>ATOM_UTF8_EXT</c></seeguide>, - <seeguide marker="#SMALL_ATOM_EXT"><c>SMALL_ATOM_UTF8_EXT</c></seeguide>, - or <seeguide marker="#ATOM_CACHE_REF"> - <c>ATOM_CACHE_REF</c></seeguide>. - Is the module that the fun is implemented in. - </p> + <p> + The module that the fun is implemented in, + <seeguide marker="#utf8_atoms">encoded as an atom</seeguide>. + </p> </item> <tag><c>OldIndex</c></tag> <item> @@ -1295,10 +1293,8 @@ This term is the encoding for external funs: <c>fun M:F/A</c>. </p> <p> - <c>Module</c> and <c>Function</c> are atoms - (encoded using <seeguide marker="#ATOM_EXT"><c>ATOM_UTF8_EXT</c></seeguide>, - <seeguide marker="#SMALL_ATOM_EXT"><c>SMALL_ATOM_UTF8_EXT</c></seeguide>, or - <seeguide marker="#ATOM_CACHE_REF"><c>ATOM_CACHE_REF</c></seeguide>). + <c>Module</c> and <c>Function</c> are + <seeguide marker="#utf8_atoms">encoded as atoms</seeguide>. </p> <p> <c>Arity</c> is an integer encoded using @@ -1377,9 +1373,9 @@ in UTF-8. </p> <p> - For more information on encoding of atoms, see the - <seeguide marker="#utf8_atoms">note on UTF-8 encoded atoms</seeguide> - in the beginning of this section. + For more information, see the + <seeguide marker="#utf8_atoms">section on encoding atoms</seeguide> + in the beginning of this page. </p> </section> @@ -1405,9 +1401,9 @@ <seeguide marker="#ATOM_UTF8_EXT"><c>ATOM_UTF8_EXT</c></seeguide>. </p> <p> - For more information on encoding of atoms, see the - <seeguide marker="#utf8_atoms">note on UTF-8 encoded atoms</seeguide> - in the beginning of this section. + For more information, see the + <seeguide marker="#utf8_atoms">section on encoding atoms</seeguide> + in the beginning of this page. </p> </section> diff --git a/erts/doc/src/erlang.xml b/erts/doc/src/erlang.xml index 3b96ed5039..26f3f768f6 100644 --- a/erts/doc/src/erlang.xml +++ b/erts/doc/src/erlang.xml @@ -30,10 +30,11 @@ <file>erlang.xml</file> </header> <module since="">erlang</module> - <modulesummary>The Erlang BIFs.</modulesummary> + <modulesummary>The Erlang BIFs and predefined types.</modulesummary> <description> - <p>By convention, most Built-In Functions (BIFs) are included - in this module. Some of the BIFs are viewed more + <p>By convention, most Built-In Functions (BIFs) and + all predefined types are included in this module. + Some of the BIFs and all of the predefined types are viewed more or less as part of the Erlang programming language and are <em>auto-imported</em>. Thus, it is not necessary to specify the module name. For example, the calls <c>atom_to_list(erlang)</c> @@ -42,6 +43,12 @@ <p>Auto-imported BIFs are listed without module prefix. BIFs listed with module prefix are not auto-imported.</p> + <p>Predefined types are listed in the + <seeerl marker="#Predefined datatypes">Predefined datatypes</seeerl> + section of this reference manual and in the + <seeguide marker="system/reference_manual:typespec">Types and Function Specifications</seeguide> + section of the Erlang Reference Manual.</p> + <p>BIFs can fail for various reasons. All BIFs fail with reason <c>badarg</c> if they are called with arguments of an incorrect type. The other reasons are described in the @@ -52,22 +59,319 @@ </description> <datatypes> + <datatype_title>Predefined datatypes</datatype_title> + <datatype> + <name name="any" n_vars="0"/> + <desc> + <p>All possible Erlang terms. Synonym for <c>term()</c>.</p> + </desc> + </datatype> + <datatype> + <name name="arity" n_vars="0"/> + <desc> + <p>The arity of a function or type.</p> + </desc> + </datatype> + <datatype> + <name name="atom" n_vars="0"/> + <desc> + <p>An Erlang <seeguide marker="system/reference_manual:data_types#atom">atom</seeguide>.</p> + </desc> + </datatype> + <datatype> + <name name="binary" n_vars="0"/> + <desc> + <p>An Erlang <seeguide marker="system/reference_manual:data_types#bit-strings-and-binaries">binary</seeguide>, + that is, a bitstring with a size divisible by 8.</p> + </desc> + </datatype> + <datatype> + <name name="bitstring" n_vars="0"/> + <desc> + <p>An Erlang <seeguide marker="system/reference_manual:data_types#bit-strings-and-binaries">bitstring</seeguide>.</p> + </desc> + </datatype> + <datatype> + <name name="boolean" n_vars="0"/> + <desc> + <p>A <seeguide marker="system/reference_manual:data_types#boolean">boolean</seeguide> value.</p> + </desc> + </datatype> + <datatype> + <name name="byte" n_vars="0"/> + <desc> + <p>A byte of data represented by an integer.</p> + </desc> + </datatype> + <datatype> + <name name="char" n_vars="0"/> + <desc> + <p>An ASCII character or a <seeerl marker="stdlib:unicode">unicode</seeerl> + codepoint presented by an integer.</p> + </desc> + </datatype> + <datatype> + <name name="float" n_vars="0"/> + <desc> + <p>An Erlang <seeguide marker="system/reference_manual:data_types#number">float</seeguide>.</p> + </desc> + </datatype> + <datatype> + <name name="function" n_vars="0"/> + <desc> + <p>An Erlang <seeguide marker="system/reference_manual:data_types#fun">fun</seeguide>.</p> + </desc> + </datatype> + <datatype> + <name name="identifier" n_vars="0"/> + <desc> + <p>An unique identifier for some entity, for example a <seeguide marker="system/reference_manual:processes">process</seeguide>, <seeguide marker="system/reference_manual:ports#ports">port</seeguide> or <seemfa marker="#monitor/2">monitor</seemfa>.</p> + </desc> + </datatype> + <datatype> + <name name="integer" n_vars="0"/> + <desc> + <p>An Erlang <seeguide marker="system/reference_manual:data_types#number">integer</seeguide>.</p> + </desc> + </datatype> + <datatype> + <name name="iodata" n_vars="0"/> + <desc> + <p>A binary or list containing bytes and/or iodata. This datatype is used to represent data + that is meant to be output using any I/O module. + For example: <seemfa marker="kernel:file#write/2">file:write/2</seemfa> or + <seemfa marker="kernel:gen_tcp#send/2">gen_tcp:send/2</seemfa>. + </p> + <p>To convert an iodata() term to binary() you can use <seemfa marker="#iolist_to_binary/1"> + iolist_to_binary/2</seemfa>. To transcode a <seetype marker="#string">string()</seetype> + or <seetype marker="stdlib:unicode#chardata">unicode:chardata()</seetype> to iodata() + you can use <seemfa marker="stdlib:unicode#characters_to_binary/1">unicode:characters_to_binary/1</seemfa>. + </p> + </desc> + </datatype> + <datatype> + <name name="iolist" n_vars="0"/> + <desc> + <p>A list containing bytes and/or iodata. This datatype is used to represent data + that is meant to be output using any I/O module. + For example: <seemfa marker="kernel:file#write/2">file:write/2</seemfa> + or <seemfa marker="kernel:gen_tcp#send/2">gen_tcp:send/2</seemfa>.</p> + <p>In most use cases you want to use <seetype marker="#iodata">iodata()</seetype> + instead of this type.</p> + </desc> + </datatype> + <datatype> + <name name="list" n_vars="0"/> + <desc> + <p>An Erlang <seeguide marker="system/reference_manual:data_types#list">list</seeguide> + containing terms of any type.</p> + </desc> + </datatype> + <datatype> + <name name="list" n_vars="1"/> + <desc> + <p>An Erlang <seeguide marker="system/reference_manual:data_types#list">list</seeguide> + containing terms of the type <c><anno>ContentType</anno></c>.</p> + </desc> + </datatype> + <datatype> + <name name="map" n_vars="0"/> + <desc> + <p>An Erlang <seeguide marker="system/reference_manual:data_types#map">map</seeguide> + containing any number of key and value associations.</p> + </desc> + </datatype> + <datatype> + <name name="maybe_improper_list" n_vars="0"/> + <desc> + <p>An Erlang <seeguide marker="system/reference_manual:data_types#list">list</seeguide> + that is not guaranteed to end with a <seetype marker="#nil">[]</seetype>, and where the + list elements can be of any type. + </p> + </desc> + </datatype> + <datatype> + <name name="maybe_improper_list" n_vars="2"/> + <desc> + <p>An Erlang <seeguide marker="system/reference_manual:data_types#list">list</seeguide>, + that is not guaranteed to end with a <seetype marker="#nil">[]</seetype>, + and where the list elements are of the type <c><anno>ContentType</anno></c>. + </p> + </desc> + </datatype> + <datatype> + <name name="mfa" n_vars="0"/> + <desc> + <p>A three-tuple representing a <c>Module:Function/Arity</c> function signature.</p> + </desc> + </datatype> + <datatype> + <name name="module" n_vars="0"/> + <desc> + <p>An Erlang module represented by an atom.</p> + </desc> + </datatype> + <datatype> + <name name="neg_integer" n_vars="0"/> + <desc> + <p>A negative integer.</p> + </desc> + </datatype> + <datatype> + <name name="nil" n_vars="0"/> + <desc> + <p>The empty <seetype marker="#list">list()</seetype>.</p> + </desc> + </datatype> + <datatype> + <name name="no_return" n_vars="0"/> + <desc> + <p>The type used to show that a function will <em>never</em> return a value, + that is it will <em>always</em> throw an exception.</p> + </desc> + </datatype> + <datatype> + <name name="node" n_vars="0"/> + <desc> + <p>An Erlang <seeguide marker="system/reference_manual:distributed#nodes">node</seeguide> + represented by an atom.</p> + </desc> + </datatype> + <datatype> + <name name="non_neg_integer" n_vars="0"/> + <desc> + <p>A non-negative integer, that is any positive integer or 0.</p> + </desc> + </datatype> + <datatype> + <name name="none" n_vars="0"/> + <desc> + <p>This type is used to show that a function will <em>never</em> return a value; + that is it will <em>always</em> throw an exception. In a spec, use <c>no_return()</c> + for the sake of clarity.</p> + </desc> + </datatype> + <datatype> + <name name="nonempty_binary" n_vars="0"/> + <desc> + <p>A <seetype marker="#binary">binary()</seetype> that contains some data.</p> + </desc> + </datatype> + <datatype> + <name name="nonempty_bitstring" n_vars="0"/> + <desc> + <p>A <seetype marker="#bitstring">bitstring()</seetype> that contains some data.</p> + </desc> + </datatype> + <datatype> + <name name="nonempty_improper_list" n_vars="2"/> + <desc> + <p>A <seetype marker="#maybe_improper_list">maybe_improper_list/2</seetype> that contains some items.</p> + </desc> + </datatype> + <datatype> + <name name="nonempty_list" n_vars="0"/> + <desc> + <p>A <seetype marker="#list">list()</seetype> that contains some items.</p> + </desc> + </datatype> + <datatype> + <name name="nonempty_list" n_vars="1"/> + <desc> + <p>A <seetype marker="#list">list(ContentType)</seetype> that contains some items.</p> + </desc> + </datatype> + <datatype> + <name name="nonempty_maybe_improper_list" n_vars="0"/> + <desc> + <p>A <seetype marker="#maybe_improper_list">maybe_improper_list()</seetype> that contains some items.</p> + </desc> + </datatype> + <datatype> + <name name="nonempty_maybe_improper_list" n_vars="2"/> + <desc> + <p>A <seetype marker="#maybe_improper_list">maybe_improper_list(ContentType, TerminationType)</seetype> + that contains some items.</p> + </desc> + </datatype> + <datatype> + <name name="nonempty_string" n_vars="0"/> + <desc> + <p>A <seetype marker="#string">string()</seetype> that contains some characters.</p> + </desc> + </datatype> + <datatype> + <name name="number" n_vars="0"/> + <desc> + <p>An Erlang <seeguide marker="system/reference_manual:data_types#number">number</seeguide>.</p> + </desc> + </datatype> + <datatype> + <name name="pid" n_vars="0"/> + <desc> + <p>An Erlang <seeguide marker="system/reference_manual:data_types#pid">process identifier</seeguide>.</p> + </desc> + </datatype> + <datatype> + <name name="port" n_vars="0"/> + <desc> + <p>An Erlang <seeguide marker="system/reference_manual:data_types#port-identifier">port identifier</seeguide>.</p> + </desc> + </datatype> + <datatype> + <name name="pos_integer" n_vars="0"/> + <desc> + <p>An integer greater than zero.</p> + </desc> + </datatype> + <datatype> + <name name="reference" n_vars="0"/> + <desc> + <p>An Erlang <seeguide marker="system/reference_manual:data_types#reference">reference</seeguide>.</p> + </desc> + </datatype> + <datatype> + <name name="string" n_vars="0"/> + <desc> + <p>A character string represented by a list of ASCII characters or unicode codepoints.</p> + </desc> + </datatype> + <datatype> + <name name="term" n_vars="0"/> + <desc> + <p>All possible Erlang terms. Synonym for <c>any()</c>.</p> + </desc> + </datatype> + <datatype> + <name name="timeout" n_vars="0"/> + <desc> + <p>A timeout value that can be passed to a <seeguide marker="system/reference_manual:expressions#receive">receive expression</seeguide>.</p> + </desc> + </datatype> + <datatype> + <name name="tuple" n_vars="0"/> + <desc> + <p>An Erlang <seeguide marker="system/reference_manual:data_types#tuple">tuple</seeguide>.</p> + </desc> + </datatype> + + <datatype_title>Other Datatypes</datatype_title> <datatype> - <name name="ext_binary"/> + <name name="ext_binary" prefix="true"/> <desc> <p>A binary data object, structured according to the Erlang external term format.</p> </desc> </datatype> <datatype> - <name name="ext_iovec"/> + <name name="ext_iovec" prefix="true"/> <desc> <p>A term of type <seetype marker="#iovec"><c>iovec()</c></seetype>, structured according to the Erlang external term format.</p> </desc> </datatype> <datatype> - <name name="iovec"/> + <name name="iovec" prefix="true"/> <desc> <p>A list of binaries. This datatype is useful to use together with <seecref marker="erl_nif#enif_inspect_iovec"> @@ -75,27 +379,27 @@ </desc> </datatype> <datatype> - <name name="message_queue_data"></name> + <name name="message_queue_data" prefix="true"/> <desc> <p>See <seeerl marker="#process_flag_message_queue_data"> <c>process_flag(message_queue_data, MQD)</c></seeerl>.</p> </desc> </datatype> <datatype> - <name name="monitor_option"></name> + <name name="monitor_option" prefix="true"/> <desc> <p>See <seemfa marker="#monitor/3"><c>monitor/3</c></seemfa>.</p> </desc> </datatype> <datatype> - <name name="timestamp"></name> + <name name="timestamp" prefix="true"/> <desc> <p>See <seemfa marker="#timestamp/0"> <c>erlang:timestamp/0</c></seemfa>.</p> </desc> </datatype> <datatype> - <name name="time_unit"></name> + <name name="time_unit" prefix="true"/> <desc> <marker id="type_time_unit"/> <p>Supported time unit representations:</p> @@ -182,7 +486,7 @@ </desc> </datatype> <datatype> - <name name="deprecated_time_unit"></name> + <name name="deprecated_time_unit" prefix="true"/> <desc><marker id="type_deprecated_time_unit"/> <p>The <seeerl marker="#type_time_unit"><c>time_unit()</c></seeerl> type also consist of the following <em>deprecated</em> symbolic @@ -204,14 +508,14 @@ </datatype> <datatype> - <name name="dist_handle"></name> + <name name="dist_handle" prefix="true"/> <desc> <p>An opaque handle identifing a distribution channel.</p> </desc> </datatype> <datatype> - <name name="nif_resource"></name> + <name name="nif_resource" prefix="true"/> <desc> <p>An opaque handle identifing a <seecref marker="erl_nif#resource_objects">NIF resource object @@ -219,13 +523,13 @@ </desc> </datatype> <datatype> - <name name="spawn_opt_option"></name> + <name name="spawn_opt_option" prefix="true"/> <desc> <p>Options for <seemfa marker="#spawn_opt/4"><c>spawn_opt()</c></seemfa>.</p> </desc> </datatype> <datatype> - <name name="priority_level"></name> + <name name="priority_level" prefix="true"/> <desc> <p>Process priority level. For more info see <seeerl marker="#process_flag_priority"><c>process_flag(priority, @@ -233,7 +537,7 @@ </desc> </datatype> <datatype> - <name name="max_heap_size"></name> + <name name="max_heap_size" prefix="true"/> <desc> <p>Process max heap size configuration. For more info see <seeerl marker="#process_flag_max_heap_size"><c>process_flag(max_heap_size, @@ -241,7 +545,7 @@ </desc> </datatype> <datatype> - <name name="message_queue_data"></name> + <name name="message_queue_data" prefix="true"/> <desc> <p>Process message queue data configuration. For more information, see <seeerl marker="#process_flag_message_queue_data"><c>process_flag(message_queue_data, @@ -250,8 +554,8 @@ </datatype> <datatype> - <name name="stacktrace"/> - <name name="stacktrace_extrainfo"/> + <name name="stacktrace" prefix="true"/> + <name name="stacktrace_extrainfo" prefix="true"/> <desc><p>An Erlang stacktrace as described by <seeguide marker="system/reference_manual:errors#stacktrace">Errors and Error Handling</seeguide> section in the Erlang Reference Manual. @@ -565,12 +869,8 @@ client(ServerPid, Request) -> valid UTF-8 sequences.</p> <note> <p>As from Erlang/OTP 20, <c>binary_to_atom(<anno>Binary</anno>, utf8)</c> - is capable of encoding any Unicode character. Earlier versions would - fail if the binary contained Unicode characters > 255. - For more information about Unicode support in atoms, see the - <seeguide marker="erl_ext_dist#utf8_atoms">note on UTF-8 - encoded atoms</seeguide> - in section "External Term Format" in the User's Guide.</p> + is capable of decoding any Unicode character. Earlier versions would + fail if the binary contained Unicode characters > 255.</p> </note> <note> <p>The number of characters that are permitted in an atom @@ -3233,10 +3533,7 @@ is_process_alive(P2Pid), <p>As from Erlang/OTP 20, <c><anno>String</anno></c> may contain any Unicode character. Earlier versions allowed only ISO-latin-1 characters as the implementation did not allow Unicode characters - above 255. For more information on Unicode support in atoms, see - <seeguide marker="erl_ext_dist#utf8_atoms">note on UTF-8 - encoded atoms</seeguide> - in section "External Term Format" in the User's Guide.</p> + above 255.</p> <note> <p>The number of characters that are permitted in an atom name is limited. The default limits can be found in the @@ -11093,7 +11390,7 @@ hello </item> <tag><c>1</c></tag> <item> - <p>This is as of Erlang/OTP 17.0 the default. It forces any floats + <p>This is as of Erlang/OTP 17.0 the <em>default</em>. It forces any floats in the term to be encoded in a more space-efficient and exact way (namely in the 64-bit IEEE format, rather than converted to a textual representation). As from Erlang/OTP R11B-4, @@ -11105,9 +11402,16 @@ hello <tag><c>2</c></tag> <item> <p>Drops usage of the latin1 atom encoding and unconditionally - use utf8 encoding for all atoms. This will be changed to the - default in a future major release of Erlang/OTP. Erlang/OTP + use utf8 encoding for all atoms. Erlang/OTP systems as of R16B can decode this representation.</p> + <note> + <p>In Erlang/OTP 26, the default <c>minor_version</c> is planned + to change from 1 to 2. See + <seeguide marker="system/general_info:upcoming_incompatibilities#atoms_be_utf8"> + Upcoming Potential Incompatibilities + </seeguide>. + </p> + </note> </item> </taglist> <p>Option <c>deterministic</c> (introduced in OTP 24.1) can diff --git a/erts/doc/src/erts_alloc.xml b/erts/doc/src/erts_alloc.xml index cd98123c5c..a5aff575f5 100644 --- a/erts/doc/src/erts_alloc.xml +++ b/erts/doc/src/erts_alloc.xml @@ -839,6 +839,27 @@ condition if the user limit on the amount of locked memory is reached.</p> </item> + <tag><marker id="Mdai"/><c><![CDATA[+Mdai max|<amount>]]></c></tag> + <item> + <p> + Set amount of dirty allocator instances used. Defaults to + <c>0</c>. That is, by default no instances will be used. + The maximum amount of instances equals the amount of dirty + CPU schedulers on the system. + </p> + <p> + By default, each normal scheduler thread has its own allocator + instance for each allocator. All other threads in the system, + including dirty schedulers, share one instance for each allocator. + By enabling dirty allocator instances, dirty schedulers will + get and use their own set of allocator instances. Note that these + instances are not exclusive to each dirty scheduler, but instead + shared among dirty schedulers. The more instances used the less + risk of lock contention on these allocator instances. Memory + consumption do however increase with increased amount of dirty + allocator instances. + </p> + </item> </taglist> </section> </section> diff --git a/erts/doc/src/notes.xml b/erts/doc/src/notes.xml index 4cffeca776..a4a6c8b0f8 100644 --- a/erts/doc/src/notes.xml +++ b/erts/doc/src/notes.xml @@ -31,6 +31,162 @@ </header> <p>This document describes the changes made to the ERTS application.</p> +<section><title>Erts 12.1.5</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + The runtime system could call <c>select()</c> with a too + large timeout value when executing on MacOS. This could + in turn cause the runtime system to crash.</p> + <p> + Own Id: OTP-17735 Aux Id: GH-5339 </p> + </item> + <item> + <p> The fix for Linux's behaviour when reconnecting an + UDP socket in PR-5120 released in OTP-24.1.2 has been + refined to only dissolve the socket's connection before a + connect if the socket is already connected, that is: only + for a reconnect. </p><p> This allows code to open a + socket with an ephemeral port, get the port number and + connect; without the port number changing (on Linux). + This turned out to have at least one valid use case + (besides test cases). </p><p> Should one reconnect the + socket then the port number may change, on Linux; it is a + known quirk, which can be worked around by binding to a + specific port number when opening the socket. If you can + do without an ephemeral port, that is... </p> + <p> + Own Id: OTP-17736 Aux Id: GH-5279, PR-5120, OTP-17559 </p> + </item> + <item> + <p>Certain distributed signals that for various reasons + must to be forced into the distribution buffer even when + it is full would instead be lost if the distribution + buffer was full when sent. The effected signals:</p> + <list> <item><c>EXIT</c> signals with exit reasons of one + word size.</item> <item><c>DOWN</c> signals with exit + reasons of one word size.</item> <item><c>demonitor</c> + signals from a terminating process.</item> + <item><c>unlink_ack</c> signals on OTP 23 and 24.</item> + <item><c>spawn_reply</c> signals on OTP 23 and 24.</item> + </list> + <p> + Own Id: OTP-17737 Aux Id: GH-5346, GH-4989 </p> + </item> + </list> + </section> + +</section> + +<section><title>Erts 12.1.4</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Fix bug where a gen_tcp write error that happened during + a delayed_send would cause a use after free segfault.</p> + <p> + Own Id: OTP-17731 Aux Id: PR-5285 </p> + </item> + <item> + <p> + Fix x86 JIT bug where a <c>rem</c> instruction could + cause a segfault if given values that would cause an + badarith exception.</p> + <p> + Own Id: OTP-17732 Aux Id: PR-5331 ERIERL-664 </p> + </item> + </list> + </section> + +</section> + +<section><title>Erts 12.1.3</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Reduction counter was not updated before and after doing + <c>apply</c> operations on the runtime system with the + jit enabled. This caused reduction counting to get out of + sync if a garbage collection was made as part of the + <c>apply</c> operation.</p> + <p> + Own Id: OTP-17675</p> + </item> + <item> + <p> + This fixes a bug in <c>erts_factory_undo</c> that caused + the heap to not be reset correctly. The + <c>erts_factory_undo</c> function is, for example, called + when a <c>binary_to_term/1</c> call fails to reset the + heap to its state before the <c>binary_to_term/1</c> + call. This can cause the heap to contain invalid terms + which potentially can cause issues (e.g., crashes) when + the whole heap is scanned.</p> + <p> + Own Id: OTP-17677</p> + </item> + <item> + <p>When attempting to construct a binary with an segment + having an illegal type for the size (e.g. an atom), there + could be an unnecessary memory allocation (and subsequent + deallocation) before the operation failed. Amended to + fail before allocating any memory for the binary.</p> + <p> + Own Id: OTP-17686</p> + </item> + <item> + <p>Fix bug in <c>persistent_term</c> when a key-value + pair contains a magic reference that is referred more + than once. Magic references are NIF resources or returned + from BIFs like <c>ets:new</c>, <c>atomics:new</c>. The + bug could cause the memory of the referred resource to be + prematurely deallocated.</p> <p>The bug also apply to + magic references in message passing on a runtime built + with configure option + <c>--enable-sharing-preserving</c>.</p> <p>Bug exist for + 64-bit since OTP-24.0 and for 32-bit since OTP-20.0.</p> + <p> + Own Id: OTP-17700 Aux Id: GH-5271, PR-5273 </p> + </item> + <item> + <p>Fixed a crash when inspecting the stack trace of an + exception raised at a very high line number.</p> + <p>This bug was introduced in OTP 24.</p> + <p> + Own Id: OTP-17712</p> + </item> + <item> + <p>The following two bugs that caused + <c>erlang:demonitor()</c> to behave erroneously have been + fixed. The bugs were only triggered if the monitor that + was removed by <c>demonitor()</c> had previously been + created simultaneously as a monitor and as an alias.</p> + <list> <item>A demonitor operation on a monitor created + using the <c>{alias, reply_demonitor}</c> option + erroneously behaved as if the <c>{alias, + explicit_unalias}</c> option had been used.</item> + <item>A demonitor operation did not prevent a + corresponding <c>'DOWN'</c> message from being delivered + if the monitor reference was kept as an active alias + after the operation. This could only occur if the + monitored process simultaneously terminated before the + demonitor signal reached it, and the exit reason was not + an immediate term. That is, a term larger than one + machine word.</item> </list> + <p> + Own Id: OTP-17722 Aux Id: GH-5310, PR-5313 </p> + </item> + </list> + </section> + +</section> + <section><title>Erts 12.1.2</title> <section><title>Improvements and New Features</title> @@ -1064,6 +1220,137 @@ </section> +<section><title>Erts 11.2.2.8</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + The runtime system could call <c>select()</c> with a too + large timeout value when executing on MacOS. This could + in turn cause the runtime system to crash.</p> + <p> + Own Id: OTP-17735 Aux Id: GH-5339 </p> + </item> + <item> + <p>Certain distributed signals that for various reasons + must to be forced into the distribution buffer even when + it is full would instead be lost if the distribution + buffer was full when sent. The effected signals:</p> + <list> <item><c>EXIT</c> signals with exit reasons of one + word size.</item> <item><c>DOWN</c> signals with exit + reasons of one word size.</item> <item><c>demonitor</c> + signals from a terminating process.</item> + <item><c>unlink_ack</c> signals on OTP 23 and 24.</item> + <item><c>spawn_reply</c> signals on OTP 23 and 24.</item> + </list> + <p> + Own Id: OTP-17737 Aux Id: GH-5346, GH-4989 </p> + </item> + </list> + </section> + +</section> + +<section><title>Erts 11.2.2.7</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + A call to <c>process_info(Pid, status)</c> could + erroneously report the status <c>running</c> when it + should have reported <c>waiting</c>. This occurred when + the calling process was executing on a higher priority + than the process being inspected. This bug has been + present since OTP 21.0 (erts version 10.0).</p> + <p> + Own Id: OTP-17628</p> + </item> + <item> + <p> + A race between an exiting port and handling of + simultaneously received signals to that port could cause + a runtime system crash. The effected signals are + <c>link</c>, <c>monitor</c> and <c>demonitor</c>. On OTP + 22 a similiar race could also cause a memory leak when + receiving an <c>unlink</c> signal.</p> + <p> + Own Id: OTP-17642 Aux Id: PR-5248 </p> + </item> + <item> + <p> + The message queue of a process entered an inconsistent + state after a <c>receive</c> expression with an invalid + timeout value was executed. If the exception raised due + to the invalid timeout value was caught, the following + <c>receive</c> expression executed by the process could + fail to match messages already present in the message + queue.</p> + <p> + On OTP 24 this could also cause the whole runtime system + to crash.</p> + <p> + Own Id: OTP-17651 Aux Id: GH-5235, PR-5244 </p> + </item> + <item> + <p> + Sending a <c>Port ! {PortOwner, close}</c> signal from a + process other than the port owner could erroneously + trigger a <c>badsig</c> exit signal being sent to the + port owner process even though the correct + <c>PortOwner</c> had been passed in the signal.</p> + <p> + Own Id: OTP-17665 Aux Id: PR-5248 </p> + </item> + <item> + <p> + This fixes a bug in <c>erts_factory_undo</c> that caused + the heap to not be reset correctly. The + <c>erts_factory_undo</c> function is, for example, called + when a <c>binary_to_term/1</c> call fails to reset the + heap to its state before the <c>binary_to_term/1</c> + call. This can cause the heap to contain invalid terms + which potentially can cause issues (e.g., crashes) when + the whole heap is scanned.</p> + <p> + Own Id: OTP-17677</p> + </item> + <item> + <p>Fix bug in <c>persistent_term</c> when a key-value + pair contains a magic reference that is referred more + than once. Magic references are NIF resources or returned + from BIFs like <c>ets:new</c>, <c>atomics:new</c>. The + bug could cause the memory of the referred resource to be + prematurely deallocated.</p> <p>The bug also apply to + magic references in message passing on a runtime built + with configure option + <c>--enable-sharing-preserving</c>.</p> <p>Bug exist for + 64-bit since OTP-24.0 and for 32-bit since OTP-20.0.</p> + <p> + Own Id: OTP-17700 Aux Id: GH-5271, PR-5273 </p> + </item> + </list> + </section> + + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + The python scripts that existed in + erts/lib_src/yielding_c_fun/lib/tiny_regex_c/scripts had + a license that was incompatible with Erlang/OTP's + license. This ticket removes these scripts that were not + used by us.</p> + <p> + Own Id: OTP-17658</p> + </item> + </list> + </section> + +</section> + <section><title>Erts 11.2.2.6</title> <section><title>Fixed Bugs and Malfunctions</title> @@ -2664,6 +2951,128 @@ </section> +<section><title>Erts 10.7.2.15</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + The runtime system could call <c>select()</c> with a too + large timeout value when executing on MacOS. This could + in turn cause the runtime system to crash.</p> + <p> + Own Id: OTP-17735 Aux Id: GH-5339 </p> + </item> + <item> + <p>Certain distributed signals that for various reasons + must to be forced into the distribution buffer even when + it is full would instead be lost if the distribution + buffer was full when sent. The effected signals:</p> + <list> <item><c>EXIT</c> signals with exit reasons of one + word size.</item> <item><c>DOWN</c> signals with exit + reasons of one word size.</item> <item><c>demonitor</c> + signals from a terminating process.</item> + <item><c>unlink_ack</c> signals on OTP 23 and 24.</item> + <item><c>spawn_reply</c> signals on OTP 23 and 24.</item> + </list> + <p> + Own Id: OTP-17737 Aux Id: GH-5346, GH-4989 </p> + </item> + </list> + </section> + +</section> + +<section><title>Erts 10.7.2.14</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p>On 32-bit computers, <c>binary_to_term/1,2</c> is now + more resilient against corrupted binaries containing maps + in the external format.</p> + <p> + Own Id: OTP-17604</p> + </item> + <item> + <p> + A call to <c>process_info(Pid, status)</c> could + erroneously report the status <c>running</c> when it + should have reported <c>waiting</c>. This occurred when + the calling process was executing on a higher priority + than the process being inspected. This bug has been + present since OTP 21.0 (erts version 10.0).</p> + <p> + Own Id: OTP-17628</p> + </item> + <item> + <p> + A race between an exiting port and handling of + simultaneously received signals to that port could cause + a runtime system crash. The effected signals are + <c>link</c>, <c>monitor</c> and <c>demonitor</c>. On OTP + 22 a similiar race could also cause a memory leak when + receiving an <c>unlink</c> signal.</p> + <p> + Own Id: OTP-17642 Aux Id: PR-5248 </p> + </item> + <item> + <p> + The message queue of a process entered an inconsistent + state after a <c>receive</c> expression with an invalid + timeout value was executed. If the exception raised due + to the invalid timeout value was caught, the following + <c>receive</c> expression executed by the process could + fail to match messages already present in the message + queue.</p> + <p> + On OTP 24 this could also cause the whole runtime system + to crash.</p> + <p> + Own Id: OTP-17651 Aux Id: GH-5235, PR-5244 </p> + </item> + <item> + <p> + Sending a <c>Port ! {PortOwner, close}</c> signal from a + process other than the port owner could erroneously + trigger a <c>badsig</c> exit signal being sent to the + port owner process even though the correct + <c>PortOwner</c> had been passed in the signal.</p> + <p> + Own Id: OTP-17665 Aux Id: PR-5248 </p> + </item> + <item> + <p> + This fixes a bug in <c>erts_factory_undo</c> that caused + the heap to not be reset correctly. The + <c>erts_factory_undo</c> function is, for example, called + when a <c>binary_to_term/1</c> call fails to reset the + heap to its state before the <c>binary_to_term/1</c> + call. This can cause the heap to contain invalid terms + which potentially can cause issues (e.g., crashes) when + the whole heap is scanned.</p> + <p> + Own Id: OTP-17677</p> + </item> + <item> + <p>Fix bug in <c>persistent_term</c> when a key-value + pair contains a magic reference that is referred more + than once. Magic references are NIF resources or returned + from BIFs like <c>ets:new</c>, <c>atomics:new</c>. The + bug could cause the memory of the referred resource to be + prematurely deallocated.</p> <p>The bug also apply to + magic references in message passing on a runtime built + with configure option + <c>--enable-sharing-preserving</c>.</p> <p>Bug exist for + 64-bit since OTP-24.0 and for 32-bit since OTP-20.0.</p> + <p> + Own Id: OTP-17700 Aux Id: GH-5271, PR-5273 </p> + </item> + </list> + </section> + +</section> + <section><title>Erts 10.7.2.13</title> <section><title>Fixed Bugs and Malfunctions</title> diff --git a/erts/emulator/asan/asan_logs_to_html b/erts/emulator/asan/asan_logs_to_html index 14c9b7fcde..b83bdb9d31 100755 --- a/erts/emulator/asan/asan_logs_to_html +++ b/erts/emulator/asan/asan_logs_to_html @@ -152,7 +152,8 @@ match_loop(Out, Bin, RegEx0, LogAcc0, PrevEnd, Unmatched0, LM0) -> %% ErrorReport "(?:(==ERROR: AddressSanitizer:.*\n" "(?:.*\n)+?)" % any lines (non-greedy) - "^(?:==|--))" % stop at line begining with == or -- + "(?:^(?:==|--)|\\z))" % stop at line begining with == or -- + % or at end-of-string "|" %% Skipped "(?:^[=-]+$)" % skip lines consisting only of = or - @@ -234,7 +235,7 @@ match_loop(Out, Bin, RegEx0, LogAcc0, PrevEnd, Unmatched0, LM0) -> LogAcc1 = case iolist_size(Unmatched1) > 500 of true -> - Txt = [io_format("<h2>WARNING!!! May be unmatched error reports" + Txt = [io_format("<h2>Unrecognized error reports" " in file ~s:</h2>\n<pre>~s</pre>", [LogAcc0#logacc.srcfile, Unmatched1])], log_error(Out, LogAcc0, Txt); diff --git a/erts/emulator/beam/beam_code.h b/erts/emulator/beam/beam_code.h index 3465c4e6ef..0ef5bf6471 100644 --- a/erts/emulator/beam/beam_code.h +++ b/erts/emulator/beam/beam_code.h @@ -29,7 +29,11 @@ #define IS_VALID_LOCATION(File, Line) \ ((unsigned) (File) < 255 && (unsigned) (Line) < ((1 << 24) - 1)) -#define MAKE_LOCATION(File, Line) (((File) << 24) | (Line)) +/* Builds a location entry, silently ignoring unrepresentable locations. */ +#define MAKE_LOCATION(File, Line) \ + (IS_VALID_LOCATION((File), (Line)) ? \ + (((File) << 24) | (Line)) : \ + LINE_INVALID_LOCATION) #define LOC_FILE(Loc) ((Loc) >> 24) #define LOC_LINE(Loc) ((Loc) & ((1 << 24)-1)) diff --git a/erts/emulator/beam/beam_debug.c b/erts/emulator/beam/beam_debug.c index a50ec46659..f8bc2af783 100644 --- a/erts/emulator/beam/beam_debug.c +++ b/erts/emulator/beam/beam_debug.c @@ -96,7 +96,7 @@ erts_debug_size_shared_1(BIF_ALIST_1) } BIF_RETTYPE -erts_debug_copy_shared_1(BIF_ALIST_1) +erts_debug_copy_shared_2(BIF_ALIST_2) { Process* p = BIF_P; Eterm term = BIF_ARG_1; @@ -106,6 +106,13 @@ erts_debug_copy_shared_1(BIF_ALIST_1) erts_shcopy_t info; INITIALIZE_SHCOPY(info); + switch (BIF_ARG_2) { + case am_true: info.copy_literals = 1; break; + case am_false: info.copy_literals = 0; break; + default: + BIF_ERROR(p, BADARG); + } + size = copy_shared_calculate(term, &info); if (size > 0) { hp = HAlloc(p, size); diff --git a/erts/emulator/beam/bif.c b/erts/emulator/beam/bif.c index c81b2a9f48..267bfc4af4 100644 --- a/erts/emulator/beam/bif.c +++ b/erts/emulator/beam/bif.c @@ -354,7 +354,6 @@ demonitor(Process *c_p, Eterm ref, Eterm *multip) return am_false; /* Not a monitor (may have been...) */ switch (mon->flags & ERTS_ML_STATE_ALIAS_MASK) { - case ERTS_ML_STATE_ALIAS_ONCE: case ERTS_ML_STATE_ALIAS_UNALIAS: { ErtsMonitorData *amdp = erts_monitor_create(ERTS_MON_TYPE_ALIAS, ref, @@ -366,6 +365,7 @@ demonitor(Process *c_p, Eterm ref, Eterm *multip) erts_monitor_tree_replace(&ERTS_P_MONITORS(c_p), mon, &amdp->origin); break; } + case ERTS_ML_STATE_ALIAS_ONCE: case ERTS_ML_STATE_ALIAS_DEMONITOR: erts_pid_ref_delete(ref); /* fall through... */ @@ -4271,7 +4271,7 @@ BIF_RETTYPE halt_2(BIF_ALIST_2) erts_exit(ERTS_ABORT_EXIT, ""); } else if (is_list(BIF_ARG_1) || BIF_ARG_1 == NIL) { -# define HALT_MSG_SIZE 200 +# define HALT_MSG_SIZE 1023 static byte halt_msg[4*HALT_MSG_SIZE+1]; Sint written; diff --git a/erts/emulator/beam/bif.tab b/erts/emulator/beam/bif.tab index 9ba49376dd..c3e40633e3 100644 --- a/erts/emulator/beam/bif.tab +++ b/erts/emulator/beam/bif.tab @@ -199,6 +199,8 @@ bif erts_internal:ets_super_user/1 bif erts_internal:spawn_request/4 bif erts_internal:dist_spawn_request/4 +bif erts_internal:no_aux_work_threads/0 + bif erlang:spawn_request_abandon/1 # Static native functions in erts_literal_area_collector @@ -665,7 +667,7 @@ bif erts_internal:purge_module/2 bif binary:split/2 bif binary:split/3 bif erts_debug:size_shared/1 -bif erts_debug:copy_shared/1 +bif erts_debug:copy_shared/2 bif erlang:has_prepared_code_on_load/1 bif maps:take/2 diff --git a/erts/emulator/beam/break.c b/erts/emulator/beam/break.c index 5527b62211..df4daff230 100644 --- a/erts/emulator/beam/break.c +++ b/erts/emulator/beam/break.c @@ -792,33 +792,6 @@ erl_crash_dump_v(char *file, int line, const char* fmt, va_list args) if (ERTS_SOMEONE_IS_CRASH_DUMPING) return; - /* Order all managed threads to block, this has to be done - first to guarantee that this is the only thread to generate - crash dump. */ - erts_thr_progress_fatal_error_block(&tpd_buf); - -#ifdef ERTS_SYS_SUSPEND_SIGNAL - /* - * We suspend all scheduler threads so that we can dump some - * data about the currently running processes and scheduler data. - * We have to be very very careful when doing this as the schedulers - * could be anywhere. - */ - sys_init_suspend_handler(); - - for (i = 0; i < erts_no_schedulers; i++) { - erts_tid_t tid = ERTS_SCHEDULER_IX(i)->tid; - if (!erts_equal_tids(tid,erts_thr_self())) - sys_thr_suspend(tid); - } - -#endif - - /* Allow us to pass certain places without locking... */ - erts_atomic32_set_mb(&erts_writing_erl_crash_dump, 1); - erts_tsd_set(erts_is_crash_dumping_key, (void *) 1); - - envsz = sizeof(env); /* ERL_CRASH_DUMP_SECONDS not set * if we have a heart port, break immediately @@ -916,6 +889,36 @@ erl_crash_dump_v(char *file, int line, const char* fmt, va_list args) time(&now); erts_cbprintf(to, to_arg, "=erl_crash_dump:0.5\n%s", ctime(&now)); + /* Order all managed threads to block, this has to be done + first to guarantee that this is the only thread to generate + crash dump. */ + erts_thr_progress_fatal_error_block(&tpd_buf); + +#ifdef ERTS_SYS_SUSPEND_SIGNAL + /* + * We suspend all scheduler threads so that we can dump some + * data about the currently running processes and scheduler data. + * We have to be very very careful when doing this as the schedulers + * could be anywhere. + * It may happen that scheduler thread is suspended while holding + * malloc lock. Therefore code running in this thread must not use + * it, or it will deadlock. ctime and fdopen calls both use malloc + * internally and must be executed prior to. + */ + sys_init_suspend_handler(); + + for (i = 0; i < erts_no_schedulers; i++) { + erts_tid_t tid = ERTS_SCHEDULER_IX(i)->tid; + if (!erts_equal_tids(tid,erts_thr_self())) + sys_thr_suspend(tid); + } + +#endif + + /* Allow us to pass certain places without locking... */ + erts_atomic32_set_mb(&erts_writing_erl_crash_dump, 1); + erts_tsd_set(erts_is_crash_dumping_key, (void *) 1); + if (file != NULL) erts_cbprintf(to, to_arg, "The error occurred in file %s, line %d\n", file, line); diff --git a/erts/emulator/beam/copy.c b/erts/emulator/beam/copy.c index 23335fe595..355e15fe7a 100644 --- a/erts/emulator/beam/copy.c +++ b/erts/emulator/beam/copy.c @@ -1743,7 +1743,7 @@ Uint copy_shared_perform_x(Eterm obj, Uint size, erts_shcopy_t *info, goto cleanup_next; } case REF_SUBTAG: - if (is_magic_ref_thing(ptr)) { + if (is_magic_ref_thing_with_hdr(ptr,hdr)) { ErtsMRefThing *mreft = (ErtsMRefThing *) ptr; erts_refc_inc(&mreft->mb->intern.refc, 2); goto off_heap_node_container_common; diff --git a/erts/emulator/beam/dist.c b/erts/emulator/beam/dist.c index e8392cd486..e0399904ed 100644 --- a/erts/emulator/beam/dist.c +++ b/erts/emulator/beam/dist.c @@ -1582,12 +1582,12 @@ erts_dsig_send_reg_msg(ErtsDSigSendContext* ctx, Eterm remote_name, /* local has died, deliver the exit signal to remote */ int -erts_dsig_send_exit_tt(ErtsDSigSendContext *ctx, Eterm local, Eterm remote, +erts_dsig_send_exit_tt(ErtsDSigSendContext *ctx, Process *c_p, Eterm remote, Eterm reason, Eterm token) { - Eterm ctl, msg = THE_NON_VALUE; + Eterm ctl, msg = THE_NON_VALUE, local = c_p->common.id; #ifdef USE_VM_PROBES - Process *sender = ctx->c_p; + Process *sender = c_p; Sint tok_label = 0; Sint tok_lastcnt = 0; Sint tok_serial = 0; @@ -1601,7 +1601,7 @@ erts_dsig_send_exit_tt(ErtsDSigSendContext *ctx, Eterm local, Eterm remote, msg = reason; if (have_seqtrace(token)) { - seq_trace_update_serial(ctx->c_p); + seq_trace_update_serial(c_p); seq_trace_output_exit(token, reason, SEQ_TRACE_SEND, remote, local); if (ctx->dep->dflags & DFLAG_EXIT_PAYLOAD) { ctl = TUPLE4(&ctx->ctl_heap[0], @@ -2706,6 +2706,7 @@ int erts_net_message(Port *prt, so.group_leader = gl; so.mfa = mfa; so.dist_entry = dep; + so.conn_id = conn_id; so.mld = ede.mld; so.edep = edep; so.ede_hfrag = ede_hfrag; @@ -2940,6 +2941,11 @@ erts_dsig_prepare(ErtsDSigSendContext *ctx, int no_trap, int connect) { + /* + * No process imply that we should force data through. That + * is, ignore busy state of dist entry and allow enqueue + * regardless of its state... + */ int res; ASSERT(no_trap || proc); @@ -2984,7 +2990,7 @@ retry: goto fail; } - if (no_suspend) { + if (no_suspend && proc) { if (erts_atomic32_read_acqb(&dep->qflgs) & ERTS_DE_QFLG_BUSY) { res = ERTS_DSIG_PREP_WOULD_SUSPEND; goto fail; diff --git a/erts/emulator/beam/dist.h b/erts/emulator/beam/dist.h index 12392955e6..7666e8951f 100644 --- a/erts/emulator/beam/dist.h +++ b/erts/emulator/beam/dist.h @@ -400,7 +400,7 @@ struct dist_sequences { extern int erts_dsig_send_msg(ErtsDSigSendContext*, Eterm, Eterm); extern int erts_dsig_send_reg_msg(ErtsDSigSendContext*, Eterm, Eterm, Eterm); extern int erts_dsig_send_link(ErtsDSigSendContext *, Eterm, Eterm); -extern int erts_dsig_send_exit_tt(ErtsDSigSendContext *, Eterm, Eterm, Eterm, Eterm); +extern int erts_dsig_send_exit_tt(ErtsDSigSendContext *, Process *, Eterm, Eterm, Eterm); extern int erts_dsig_send_unlink(ErtsDSigSendContext *, Eterm, Eterm, Uint64); extern int erts_dsig_send_unlink_ack(ErtsDSigSendContext *, Eterm, Eterm, Uint64); extern int erts_dsig_send_group_leader(ErtsDSigSendContext *, Eterm, Eterm); diff --git a/erts/emulator/beam/emu/bs_instrs.tab b/erts/emulator/beam/emu/bs_instrs.tab index c52367de2a..84adc06a9d 100644 --- a/erts/emulator/beam/emu/bs_instrs.tab +++ b/erts/emulator/beam/emu/bs_instrs.tab @@ -795,7 +795,7 @@ i_bs_utf16_size(Src, Dst) { $Dst = result; } -bs_put_utf16(Fail, Flags, Src) { +i_bs_put_utf16(Fail, Flags, Src) { if (!erts_bs_put_utf16(ERL_BITS_ARGS_2($Src, $Flags))) { $BADARG($Fail); } diff --git a/erts/emulator/beam/emu/msg_instrs.tab b/erts/emulator/beam/emu/msg_instrs.tab index 085cdb34cd..15002fa807 100644 --- a/erts/emulator/beam/emu/msg_instrs.tab +++ b/erts/emulator/beam/emu/msg_instrs.tab @@ -271,8 +271,7 @@ remove_message() { tok_label, tok_lastcnt, tok_serial); } #endif - erts_msgq_unlink_msg(c_p, msgp); - erts_msgq_set_save_first(c_p); + erts_msgq_unlink_msg_set_save_first(c_p, msgp); CANCEL_TIMER(c_p); erts_save_message_in_proc(c_p, msgp); diff --git a/erts/emulator/beam/emu/ops.tab b/erts/emulator/beam/emu/ops.tab index 336c4d0283..e1dba18a9f 100644 --- a/erts/emulator/beam/emu/ops.tab +++ b/erts/emulator/beam/emu/ops.tab @@ -1203,8 +1203,8 @@ i_bs_get_utf8 xy f? d bs_skip_utf8 Fail=f Ms=xy u u => i_bs_get_utf8 Ms Fail x -bs_get_utf16 Fail=f Ms=xy u Flags=u Dst=d => i_bs_get_utf16 Ms Fail Flags Dst -bs_skip_utf16 Fail=f Ms=xy u Flags=u => i_bs_get_utf16 Ms Fail Flags x +bs_get_utf16 Fail=f Ms=xy u Flags=u Dst=d => get_utf16(Fail, Ms, Flags, Dst) +bs_skip_utf16 Fail=f Ms=xy Unit=u Flags=u => bs_get_utf16 Fail Ms Unit Flags x i_bs_get_utf16 xy f? t d @@ -1303,6 +1303,8 @@ bs_utf16_size j Src Dst=d => i_bs_utf16_size Src Dst bs_put_utf8 Fail u Src => i_bs_put_utf8 Fail Src +bs_put_utf16 Fail Flags Src => put_utf16(Fail, Flags, Src) + bs_put_utf32 Fail=j Flags=u Src=s => \ i_bs_validate_unicode Fail Src | bs_put_integer Fail i=32 u=1 Flags Src @@ -1310,15 +1312,15 @@ i_bs_utf8_size S x i_bs_utf16_size S x i_bs_put_utf8 j? S -bs_put_utf16 j? t S +i_bs_put_utf16 j? t S i_bs_validate_unicode j? S -# Handle unoptimized code. +# Handle unoptimized code and the 'native' flag for utf16 segments. i_bs_utf8_size Src=c Dst => move Src x | i_bs_utf8_size x Dst i_bs_utf16_size Src=c Dst => move Src x | i_bs_utf16_size x Dst i_bs_put_utf8 Fail Src=c => move Src x | i_bs_put_utf8 Fail x -bs_put_utf16 Fail Flags Src=c => move Src x | bs_put_utf16 Fail Flags x +i_bs_put_utf16 Fail Flags Src=c => move Src x | i_bs_put_utf16 Fail Flags x i_bs_validate_unicode Fail Src=c => move Src x | i_bs_validate_unicode Fail x # diff --git a/erts/emulator/beam/erl_alloc.c b/erts/emulator/beam/erl_alloc.c index 2cea68a817..125a5a240e 100644 --- a/erts/emulator/beam/erl_alloc.c +++ b/erts/emulator/beam/erl_alloc.c @@ -148,6 +148,9 @@ ERTS_SCHED_PREF_QUICK_ALLOC_IMPL(aireq, ERTS_ALC_T_AINFO_REQ) ErtsAlcType_t erts_fix_core_allocator_ix; +erts_tsd_key_t erts_thr_alloc_data_key; + +Uint ERTS_WRITE_UNLIKELY(erts_no_dirty_alloc_instances); struct au_init { int enable; @@ -186,6 +189,7 @@ typedef struct { #endif int trim_threshold; int top_pad; + int dirty_alloc_insts; AlcUInit_t alloc_util; struct { char *mtrace; @@ -461,7 +465,7 @@ set_default_test_alloc_opts(struct au_init *ip) { SET_DEFAULT_ALLOC_OPTS(ip); ip->enable = 0; /* Disabled by default */ - ip->thr_spec = -1 * erts_no_schedulers; + ip->thr_spec = -1; ip->astrat = ERTS_ALC_S_FIRSTFIT; ip->init.aoff.crr_order = FF_AOFF; ip->init.aoff.blk_order = FF_BF; @@ -488,10 +492,10 @@ set_default_test_alloc_opts(struct au_init *ip) static void -adjust_tpref(struct au_init *ip, int no_sched) +adjust_tpref(struct au_init *ip, int no_sched, int no_dirty_inst) { if (ip->thr_spec) { - ip->thr_spec = no_sched; + ip->thr_spec = no_sched + no_dirty_inst; ip->thr_spec *= -1; /* thread preferred */ /* If default ... */ @@ -607,6 +611,7 @@ erts_alloc_init(int *argc, char **argv, ErtsAllocInitOpts *eaiop) #endif ERTS_DEFAULT_TRIM_THRESHOLD, ERTS_DEFAULT_TOP_PAD, + 0, /* Default dirty alloc instances */ ERTS_DEFAULT_ALCU_INIT, }; size_t fix_type_sizes[ERTS_ALC_NO_FIXED_SIZES] = {0}; @@ -644,6 +649,8 @@ erts_alloc_init(int *argc, char **argv, ErtsAllocInitOpts *eaiop) hdbg_init(); #endif + erts_tsd_key_create(&erts_thr_alloc_data_key, "erts_alc_data_key"); + lock_all_physical_memory = 0; ncpu = eaiop->ncpu; @@ -686,6 +693,7 @@ erts_alloc_init(int *argc, char **argv, ErtsAllocInitOpts *eaiop) #endif } + erts_no_dirty_alloc_instances = init.dirty_alloc_insts; /* Make adjustments for carrier migration support */ init.temp_alloc.init.util.acul = 0; @@ -729,18 +737,19 @@ erts_alloc_init(int *argc, char **argv, ErtsAllocInitOpts *eaiop) /* Only temp_alloc can use thread specific interface */ if (init.temp_alloc.thr_spec) - init.temp_alloc.thr_spec = erts_no_schedulers; + init.temp_alloc.thr_spec = erts_no_schedulers + init.dirty_alloc_insts; /* Others must use thread preferred interface */ - adjust_tpref(&init.sl_alloc, erts_no_schedulers); - adjust_tpref(&init.std_alloc, erts_no_schedulers); - adjust_tpref(&init.ll_alloc, erts_no_schedulers); - adjust_tpref(&init.eheap_alloc, erts_no_schedulers); - adjust_tpref(&init.binary_alloc, erts_no_schedulers); - adjust_tpref(&init.ets_alloc, erts_no_schedulers); - adjust_tpref(&init.driver_alloc, erts_no_schedulers); - adjust_tpref(&init.fix_alloc, erts_no_schedulers); - adjust_tpref(&init.literal_alloc, erts_no_schedulers); + adjust_tpref(&init.sl_alloc, erts_no_schedulers, init.dirty_alloc_insts); + adjust_tpref(&init.std_alloc, erts_no_schedulers, init.dirty_alloc_insts); + adjust_tpref(&init.ll_alloc, erts_no_schedulers, init.dirty_alloc_insts); + adjust_tpref(&init.eheap_alloc, erts_no_schedulers, init.dirty_alloc_insts); + adjust_tpref(&init.binary_alloc, erts_no_schedulers, init.dirty_alloc_insts); + adjust_tpref(&init.ets_alloc, erts_no_schedulers, init.dirty_alloc_insts); + adjust_tpref(&init.driver_alloc, erts_no_schedulers, init.dirty_alloc_insts); + adjust_tpref(&init.fix_alloc, erts_no_schedulers, init.dirty_alloc_insts); + adjust_tpref(&init.literal_alloc, erts_no_schedulers, init.dirty_alloc_insts); + adjust_tpref(&init.test_alloc, erts_no_schedulers, init.dirty_alloc_insts); /* @@ -756,6 +765,7 @@ erts_alloc_init(int *argc, char **argv, ErtsAllocInitOpts *eaiop) refuse_af_strategy(&init.driver_alloc); refuse_af_strategy(&init.fix_alloc); refuse_af_strategy(&init.literal_alloc); + refuse_af_strategy(&init.test_alloc); if (!init.temp_alloc.thr_spec) refuse_af_strategy(&init.temp_alloc); @@ -763,6 +773,7 @@ erts_alloc_init(int *argc, char **argv, ErtsAllocInitOpts *eaiop) erts_mtrace_pre_init(); #if HAVE_ERTS_MSEG init.mseg.nos = erts_no_schedulers; + init.mseg.ndai = init.dirty_alloc_insts; erts_mseg_init(&init.mseg); #endif @@ -1038,12 +1049,12 @@ start_au_allocator(ErtsAlcType_t alctr_n, if (!as0) continue; if (init->thr_spec < 0) { - init->init.util.ts = i == 0; + init->init.util.ts = (i == 0 || erts_no_schedulers < i); init->init.util.tspec = 0; init->init.util.tpref = -1*init->thr_spec + 1; } else { - if (i != 0) + if (0 < i && i <= erts_no_schedulers) init->init.util.ts = 0; else { if (astrat == ERTS_ALC_S_AFIT) @@ -1749,6 +1760,24 @@ handle_args(int *argc, char **argv, erts_alc_hndl_args_init_t *init) } bad_param(param, param+2); break; + case 'd': + if (has_prefix("ai", param+2)) { + arg = get_value(argv[i]+5, argv, &i); + if (sys_strcmp("max", arg) == 0) + init->dirty_alloc_insts = (int) erts_no_dirty_cpu_schedulers; + else { + Sint tmp; + char *rest; + errno = 0; + tmp = (Sint) ErtsStrToSint(arg, &rest, 10); + if (errno != 0 || rest == arg || tmp < 0) + bad_value(param, param+4, arg); + init->dirty_alloc_insts = (int) tmp; + } + break; + } + bad_param(param, param+2); + break; case 'u': if (has_prefix("ycs", argv[i]+3)) { init->alloc_util.ycs @@ -1817,6 +1846,10 @@ handle_args(int *argc, char **argv, erts_alc_hndl_args_init_t *init) argv[j++] = argv[i]; } *argc = j; + + if (init->dirty_alloc_insts > erts_no_dirty_cpu_schedulers) + init->dirty_alloc_insts = (int) erts_no_dirty_cpu_schedulers; + } static char *type_no_str(ErtsAlcType_t n) @@ -1837,46 +1870,92 @@ void erts_alloc_register_scheduler(void *vesdp) { ErtsSchedulerData *esdp = (ErtsSchedulerData *) vesdp; - int ix = (int) esdp->no; + int ix; int aix; + int normal_sched; - ASSERT(!ERTS_SCHEDULER_IS_DIRTY(esdp)); + ASSERT(esdp == erts_get_scheduler_data()); + + if (!ERTS_SCHEDULER_IS_DIRTY(esdp)) { + ix = (int) esdp->no; + ASSERT(0 < ix && ix <= erts_no_schedulers); + normal_sched = !0; + } + else if (!erts_no_dirty_alloc_instances) { + ix = 0; + normal_sched = 0; + } + else { + ix = (int) esdp->dirty_no; + ASSERT(ix > 0); + ix = ((ix - 1) % erts_no_dirty_alloc_instances) + 1; + ix += erts_no_schedulers; + ASSERT(erts_no_schedulers < ix + && ix <= (erts_no_schedulers + + erts_no_dirty_alloc_instances)); + normal_sched = 0; + } + + esdp->aux_work_data.alloc_data.delayed_dealloc_handler = normal_sched; + esdp->aux_work_data.alloc_data.alc_ix = ix; for (aix = ERTS_ALC_A_MIN; aix <= ERTS_ALC_A_MAX; aix++) { ErtsAllocatorThrSpec_t *tspec = &erts_allctr_thr_spec[aix]; - esdp->alloc_data.deallctr[aix] = NULL; - esdp->alloc_data.pref_ix[aix] = -1; - if (tspec->enabled) { - if (!tspec->dd) - esdp->alloc_data.pref_ix[aix] = ix; - else { - Allctr_t *allctr = tspec->allctr[ix]; - ASSERT(allctr); - esdp->alloc_data.deallctr[aix] = allctr; - esdp->alloc_data.pref_ix[aix] = ix; - } - } + esdp->aux_work_data.alloc_data.deallctr[aix] = NULL; + if (!normal_sched) + continue; + /* + * Delayed dealloc is handled by normal schedulers, + * but not by dirty schedulers... + */ + if (tspec->enabled && tspec->dd) { + Allctr_t *allctr = tspec->allctr[ix]; + ASSERT(allctr); + esdp->aux_work_data.alloc_data.deallctr[aix] = allctr; + } } + erts_tsd_set(erts_thr_alloc_data_key, (void *) &esdp->aux_work_data.alloc_data); } void -erts_alloc_scheduler_handle_delayed_dealloc(void *vesdp, - int *need_thr_progress, - ErtsThrPrgrVal *thr_prgr_p, - int *more_work) +erts_alloc_register_delayed_dealloc_handler_thread(ErtsThrAllocData *tadp, int ix) { - ErtsSchedulerData *esdp = (ErtsSchedulerData *) vesdp; int aix; + + /* + * Should not be a scheduler of any kind and should not + * handle an 'ix' reserved for normal schedulers... + */ + ASSERT(!erts_get_scheduler_data()); + ASSERT(0 == ix /* Aux thread... */ + || /* Handler thread for dirty scheds instances... */ + (erts_no_schedulers < ix + && ix <= (erts_no_schedulers + + erts_no_dirty_alloc_instances))); + + tadp->delayed_dealloc_handler = !0; + tadp->alc_ix = ix; for (aix = ERTS_ALC_A_MIN; aix <= ERTS_ALC_A_MAX; aix++) { - Allctr_t *allctr; - if (esdp) - allctr = esdp->alloc_data.deallctr[aix]; - else { - ErtsAllocatorThrSpec_t *tspec = &erts_allctr_thr_spec[aix]; - if (tspec->enabled && tspec->dd) - allctr = tspec->allctr[0]; - else - allctr = NULL; - } + ErtsAllocatorThrSpec_t *tspec = &erts_allctr_thr_spec[aix]; + tadp->deallctr[aix] = NULL; + if (tspec->enabled && tspec->dd) { + Allctr_t *allctr = tspec->allctr[ix]; + ASSERT(allctr); + tadp->deallctr[aix] = allctr; + } + } + erts_tsd_set(erts_thr_alloc_data_key, (void *) tadp); +} + +void +erts_alloc_handle_delayed_dealloc(ErtsThrAllocData *thr_alloc_data, + int *need_thr_progress, + ErtsThrPrgrVal *thr_prgr_p, + int *more_work) +{ + int aix; + ASSERT(thr_alloc_data); + for (aix = ERTS_ALC_A_MIN; aix <= ERTS_ALC_A_MAX; aix++) { + Allctr_t *allctr = thr_alloc_data->deallctr[aix]; if (allctr) { erts_alcu_check_delayed_dealloc(allctr, 1, @@ -1912,7 +1991,7 @@ erts_alloc_get_verify_unused_temp_alloc(Allctr_t **allctr) if (erts_allctrs_info[ERTS_ALC_A_TEMPORARY].alloc_util && erts_allctrs_info[ERTS_ALC_A_TEMPORARY].thr_spec) { ErtsAllocatorThrSpec_t *tspec; - int ix = ERTS_ALC_GET_THR_IX(); + int ix = erts_get_thr_alloc_ix(); tspec = &erts_allctr_thr_spec[ERTS_ALC_A_TEMPORARY]; if (ix < tspec->size) { @@ -3001,8 +3080,8 @@ static void reply_alloc_info(void *vair) { ErtsAllocInfoReq *air = (ErtsAllocInfoReq *) vair; - Uint sched_id = erts_get_scheduler_id(); - int global_instances = air->req_sched == sched_id; + int tix = erts_get_thr_alloc_ix(); + int global_instances = air->req_sched == tix; ErtsProcLocks rp_locks; Process *rp = air->proc; Eterm ref_copy = NIL, ai_list, msg = NIL; @@ -3028,7 +3107,7 @@ reply_alloc_info(void *vair) ? erts_alcu_sz_info : erts_alcu_info); - rp_locks = air->req_sched == sched_id ? ERTS_PROC_LOCK_MAIN : 0; + rp_locks = air->req_sched == tix ? ERTS_PROC_LOCK_MAIN : 0; sz = 0; hpp = NULL; @@ -3196,11 +3275,11 @@ reply_alloc_info(void *vair) case ERTS_ALC_INFO_A_MSEG_ALLOC: #if HAVE_ERTS_MSEG alloc_atom = erts_bld_atom(hpp, szp, "mseg_alloc"); - ainfo = erts_mseg_info(sched_id, NULL, NULL, + ainfo = erts_mseg_info(tix, NULL, NULL, hpp != NULL, air->only_sz, hpp, szp); ainfo = erts_bld_tuple(hpp, szp, 3, alloc_atom, - make_small(sched_id), + make_small(tix), ainfo); ai_list = erts_bld_cons(hpp, szp, ainfo, ai_list); #endif @@ -3209,7 +3288,7 @@ reply_alloc_info(void *vair) if (erts_allctrs_info[ai].thr_spec) { alloc_atom = erts_bld_atom(hpp, szp, (char *) ERTS_ALC_A2AD(ai)); - allctr = erts_allctr_thr_spec[ai].allctr[sched_id]; + allctr = erts_allctr_thr_spec[ai].allctr[tix]; ainfo = info_func(allctr, air->internal, hpp != NULL, NULL, NULL, hpp, szp); ai_list = erts_bld_cons(hpp, szp, @@ -3217,7 +3296,7 @@ reply_alloc_info(void *vair) hpp, szp, 3, alloc_atom, - make_small(sched_id), + make_small(tix), ainfo), ai_list); } @@ -3226,7 +3305,7 @@ reply_alloc_info(void *vair) msg = erts_bld_tuple(hpp, szp, 3, ref_copy, - make_small(sched_id), + make_small(tix), ai_list); } @@ -3245,7 +3324,7 @@ reply_alloc_info(void *vair) erts_queue_message(rp, rp_locks, mp, msg, am_system); - if (air->req_sched == sched_id) + if (air->req_sched == tix) rp_locks &= ~ERTS_PROC_LOCK_MAIN; erts_proc_unlock(rp, rp_locks); @@ -3325,13 +3404,14 @@ erts_request_alloc_info(struct process *c_p, air->allocs[airix] = ERTS_ALC_A_INVALID; erts_atomic32_init_nob(&air->refc, - (erts_aint32_t) erts_no_schedulers); + (erts_aint32_t) erts_no_aux_work_threads-1); - erts_proc_add_refc(c_p, (Sint) erts_no_schedulers); + erts_proc_add_refc(c_p, (Sint) erts_no_aux_work_threads-1); - if (erts_no_schedulers > 1) + if (erts_no_aux_work_threads > 2) erts_schedule_multi_misc_aux_work(1, - erts_no_schedulers, + 1, + erts_no_aux_work_threads-1, reply_alloc_info, (void *) air); diff --git a/erts/emulator/beam/erl_alloc.h b/erts/emulator/beam/erl_alloc.h index 831e7ab0a7..dffcef7370 100644 --- a/erts/emulator/beam/erl_alloc.h +++ b/erts/emulator/beam/erl_alloc.h @@ -107,10 +107,9 @@ typedef struct { typedef struct { Allctr_t *deallctr[ERTS_ALC_A_MAX+1]; - int pref_ix[ERTS_ALC_A_MAX+1]; - int flist_ix[ERTS_ALC_A_MAX+1]; - int pre_alc_ix; -} ErtsSchedAllocData; + int delayed_dealloc_handler; + int alc_ix; +} ErtsThrAllocData; void erts_alloc_init(int *argc, char **argv, ErtsAllocInitOpts *eaiop); void erts_alloc_late_init(void); @@ -147,11 +146,14 @@ typedef struct { void *extra; } ErtsAllocatorFunctions_t; +extern erts_tsd_key_t erts_thr_alloc_data_key; extern ErtsAllocatorFunctions_t ERTS_WRITE_UNLIKELY(erts_allctrs[ERTS_ALC_A_MAX+1]); extern ErtsAllocatorInfo_t ERTS_WRITE_UNLIKELY(erts_allctrs_info[ERTS_ALC_A_MAX+1]); +extern Uint ERTS_WRITE_UNLIKELY(erts_no_dirty_alloc_instances); + typedef struct { int enabled; int dd; @@ -175,10 +177,12 @@ void erts_allctr_wrapper_pre_lock(void); void erts_allctr_wrapper_pre_unlock(void); void erts_alloc_register_scheduler(void *vesdp); -void erts_alloc_scheduler_handle_delayed_dealloc(void *vesdp, - int *need_thr_progress, - ErtsThrPrgrVal *thr_prgr_p, - int *more_work); +void erts_alloc_register_delayed_dealloc_handler_thread(ErtsThrAllocData *tadp, + int ix); +void erts_alloc_handle_delayed_dealloc(ErtsThrAllocData *thr_alloc_data, + int *need_thr_progress, + ErtsThrPrgrVal *thr_prgr_p, + int *more_work); erts_aint32_t erts_alloc_fix_alloc_shrink(int ix, erts_aint32_t flgs); __decl_noreturn void erts_alloc_enomem(ErtsAlcType_t,Uint) @@ -230,6 +234,8 @@ int erts_is_allctr_wrapper_prelocked(void); #ifdef ERTS_HAVE_IS_IN_LITERAL_RANGE int erts_is_in_literal_range(void* ptr); #endif +ErtsThrAllocData *erts_get_thr_alloc_data(void); +int erts_get_thr_alloc_ix(void); #endif /* #if !ERTS_ALC_DO_INLINE */ @@ -321,6 +327,21 @@ int erts_is_allctr_wrapper_prelocked(void) && !!erts_tsd_get(erts_allctr_prelock_tsd_key); /* by me */ } +ERTS_ALC_INLINE +ErtsThrAllocData *erts_get_thr_alloc_data(void) +{ + return (ErtsThrAllocData *) erts_tsd_get(erts_thr_alloc_data_key); +} + +ERTS_ALC_INLINE +int erts_get_thr_alloc_ix(void) +{ + ErtsThrAllocData *tadp = (ErtsThrAllocData *) erts_tsd_get(erts_thr_alloc_data_key); + if (!tadp) + return 0; + return tadp->alc_ix; +} + #ifdef ERTS_HAVE_IS_IN_LITERAL_RANGE ERTS_ALC_FORCE_INLINE @@ -345,8 +366,6 @@ int erts_is_in_literal_range(void* ptr) #endif /* #if ERTS_ALC_DO_INLINE || defined(ERTS_ALC_INTERNAL__) */ -#define ERTS_ALC_GET_THR_IX() ((int) erts_get_scheduler_id()) - typedef void (*erts_alloc_verify_func_t)(Allctr_t *); erts_alloc_verify_func_t diff --git a/erts/emulator/beam/erl_alloc_util.c b/erts/emulator/beam/erl_alloc_util.c index ac3e9fe967..0c405a87b8 100644 --- a/erts/emulator/beam/erl_alloc_util.c +++ b/erts/emulator/beam/erl_alloc_util.c @@ -1715,7 +1715,7 @@ get_pref_allctr(void *extra) ErtsAllocatorThrSpec_t *tspec = (ErtsAllocatorThrSpec_t *) extra; int pref_ix; - pref_ix = ERTS_ALC_GET_THR_IX(); + pref_ix = erts_get_thr_alloc_ix(); ERTS_CT_ASSERT(sizeof(UWord) == sizeof(Allctr_t *)); ASSERT(0 <= pref_ix && pref_ix < tspec->size); @@ -6047,7 +6047,7 @@ erts_alcu_alloc_thr_spec(ErtsAlcType_t type, void *extra, Uint size) Allctr_t *allctr; void *res; - ix = ERTS_ALC_GET_THR_IX(); + ix = erts_get_thr_alloc_ix(); ASSERT(0 <= ix && ix < tspec->size); @@ -6174,7 +6174,7 @@ erts_alcu_free_thr_spec(ErtsAlcType_t type, void *extra, void *p) int ix; Allctr_t *allctr; - ix = ERTS_ALC_GET_THR_IX(); + ix = erts_get_thr_alloc_ix(); ASSERT(0 <= ix && ix < tspec->size); @@ -6460,7 +6460,7 @@ erts_alcu_realloc_thr_spec(ErtsAlcType_t type, void *extra, Allctr_t *allctr; void *res; - ix = ERTS_ALC_GET_THR_IX(); + ix = erts_get_thr_alloc_ix(); ASSERT(0 <= ix && ix < tspec->size); @@ -6497,7 +6497,7 @@ erts_alcu_realloc_mv_thr_spec(ErtsAlcType_t type, void *extra, Allctr_t *allctr; void *res; - ix = ERTS_ALC_GET_THR_IX(); + ix = erts_get_thr_alloc_ix(); ASSERT(0 <= ix && ix < tspec->size); @@ -7371,7 +7371,7 @@ static int blockscan_sweep_cpool(blockscan_t *state) } static int blockscan_get_specific_allocator(int allocator_num, - int sched_id, + int aux_work_tid, Allctr_t **out) { ErtsAllocatorInfo_t *ai; @@ -7379,7 +7379,7 @@ static int blockscan_get_specific_allocator(int allocator_num, ASSERT(allocator_num >= ERTS_ALC_A_MIN && allocator_num <= ERTS_ALC_A_MAX); - ASSERT(sched_id >= 0 && sched_id <= erts_no_schedulers); + ASSERT(0 <= aux_work_tid && aux_work_tid < erts_no_aux_work_threads); ai = &erts_allctrs_info[allocator_num]; @@ -7388,7 +7388,7 @@ static int blockscan_get_specific_allocator(int allocator_num, } if (!ai->thr_spec) { - if (sched_id != 0) { + if (aux_work_tid != 0) { /* Only thread-specific allocators can be scanned on a specific * scheduler. */ return 0; @@ -7399,9 +7399,9 @@ static int blockscan_get_specific_allocator(int allocator_num, } else { ErtsAllocatorThrSpec_t *tspec = (ErtsAllocatorThrSpec_t*)ai->extra; - ASSERT(sched_id < tspec->size); + ASSERT(aux_work_tid < tspec->size); - allocator = tspec->allctr[sched_id]; + allocator = tspec->allctr[aux_work_tid]; } *out = allocator; @@ -7411,14 +7411,9 @@ static int blockscan_get_specific_allocator(int allocator_num, static void blockscan_sched_trampoline(void *arg) { - ErtsAlcuBlockscanYieldData *yield; - ErtsSchedulerData *esdp; - blockscan_t *scanner; - - esdp = erts_get_scheduler_data(); - scanner = (blockscan_t*)arg; - - yield = ERTS_SCHED_AUX_YIELD_DATA(esdp, alcu_blockscan); + ErtsAuxWorkData *awdp = erts_get_aux_work_data(); + ErtsAlcuBlockscanYieldData *yield = &awdp->yield.alcu_blockscan; + blockscan_t *scanner = (blockscan_t*)arg; ASSERT((yield->last == NULL) == (yield->current == NULL)); @@ -7435,20 +7430,14 @@ static void blockscan_sched_trampoline(void *arg) scanner->scanner_queue = NULL; yield->last = scanner; - erts_notify_new_aux_yield_work(esdp); + erts_more_yield_aux_work(awdp); } static void blockscan_dispatch(blockscan_t *scanner, Process *owner, - Allctr_t *allocator, int sched_id) + Allctr_t *allocator, int aux_work_tid) { ASSERT(erts_get_scheduler_id() != 0); - if (sched_id == 0) { - /* Global instances are always handled on the current scheduler. */ - sched_id = ERTS_ALC_GET_THR_IX(); - ASSERT(allocator->thread_safe); - } - scanner->allocator = allocator; scanner->process = owner; @@ -7464,22 +7453,20 @@ static void blockscan_dispatch(blockscan_t *scanner, Process *owner, scanner->next_op = blockscan_sweep_mbcs; } - /* Aux yield jobs can only be set up while running on the scheduler that - * services them, so we move there before continuing. + /* Aux yield jobs can only be set up while running on the aux work + * thread that services them, so we move there before continuing. * - * We can't drive the scan itself through this since the scheduler will + * We can't drive the scan itself through this since the aux work thread will * always finish *all* misc aux work in one go which makes it impossible to * yield. */ - erts_schedule_misc_aux_work(sched_id, blockscan_sched_trampoline, scanner); + erts_schedule_misc_aux_work(aux_work_tid, blockscan_sched_trampoline, scanner); } -int erts_handle_yielded_alcu_blockscan(ErtsSchedulerData *esdp, - ErtsAlcuBlockscanYieldData *yield) +int erts_handle_yielded_alcu_blockscan(ErtsAuxWorkData *awdp) { + ErtsAlcuBlockscanYieldData *yield = &awdp->yield.alcu_blockscan; blockscan_t *scanner = yield->current; - (void)esdp; - ASSERT((yield->last == NULL) == (yield->current == NULL)); if (scanner) { @@ -7508,14 +7495,10 @@ int erts_handle_yielded_alcu_blockscan(ErtsSchedulerData *esdp, return 0; } -void erts_alcu_sched_spec_data_init(ErtsSchedulerData *esdp) +void erts_alcu_blockscan_init(ErtsAuxWorkData *awdp) { - ErtsAlcuBlockscanYieldData *yield; - - yield = ERTS_SCHED_AUX_YIELD_DATA(esdp, alcu_blockscan); - - yield->current = NULL; - yield->last = NULL; + awdp->yield.alcu_blockscan.current = NULL; + awdp->yield.alcu_blockscan.last = NULL; } /* ------------------------------------------------------------------------- */ @@ -7815,7 +7798,7 @@ static void gather_ahist_abort(void *arg) } int erts_alcu_gather_alloc_histograms(Process *p, int allocator_num, - int sched_id, int hist_width, + int aux_work_tid, int hist_width, UWord hist_start, Eterm ref) { gather_ahist_t *gather_state; @@ -7825,7 +7808,7 @@ int erts_alcu_gather_alloc_histograms(Process *p, int allocator_num, ASSERT(is_internal_ref(ref)); if (!blockscan_get_specific_allocator(allocator_num, - sched_id, + aux_work_tid, &allocator)) { return 0; } @@ -7846,7 +7829,7 @@ int erts_alcu_gather_alloc_histograms(Process *p, int allocator_num, gather_state->hist_slot_count = hist_width; gather_state->process = p; - blockscan_dispatch(scanner, p, allocator, sched_id); + blockscan_dispatch(scanner, p, allocator, aux_work_tid); return 1; } @@ -8138,7 +8121,7 @@ static void gather_cinfo_abort(void *arg) } int erts_alcu_gather_carrier_info(struct process *p, int allocator_num, - int sched_id, int hist_width, + int aux_work_tid, int hist_width, UWord hist_start, Eterm ref) { gather_cinfo_t *gather_state; @@ -8148,7 +8131,7 @@ int erts_alcu_gather_carrier_info(struct process *p, int allocator_num, ASSERT(is_internal_ref(ref)); if (!blockscan_get_specific_allocator(allocator_num, - sched_id, + aux_work_tid, &allocator)) { return 0; } @@ -8170,7 +8153,7 @@ int erts_alcu_gather_carrier_info(struct process *p, int allocator_num, gather_state->hist_slot_count = hist_width; gather_state->process = p; - blockscan_dispatch(scanner, p, allocator, sched_id); + blockscan_dispatch(scanner, p, allocator, aux_work_tid); return 1; } diff --git a/erts/emulator/beam/erl_alloc_util.h b/erts/emulator/beam/erl_alloc_util.h index 45e5bfc211..9444a4e32b 100644 --- a/erts/emulator/beam/erl_alloc_util.h +++ b/erts/emulator/beam/erl_alloc_util.h @@ -259,9 +259,10 @@ typedef struct { struct alcu_blockscan *last; } ErtsAlcuBlockscanYieldData; -int erts_handle_yielded_alcu_blockscan(struct ErtsSchedulerData_ *esdp, - ErtsAlcuBlockscanYieldData *yield); -void erts_alcu_sched_spec_data_init(struct ErtsSchedulerData_ *esdp); +struct ErtsAuxWorkData_; + +int erts_handle_yielded_alcu_blockscan(struct ErtsAuxWorkData_ *awdp); +void erts_alcu_blockscan_init(struct ErtsAuxWorkData_ *awdp); #endif /* !ERL_ALLOC_UTIL__ */ diff --git a/erts/emulator/beam/erl_bif_info.c b/erts/emulator/beam/erl_bif_info.c index 8466f88fd5..17141f3551 100644 --- a/erts/emulator/beam/erl_bif_info.c +++ b/erts/emulator/beam/erl_bif_info.c @@ -5127,7 +5127,7 @@ static BIF_RETTYPE gather_histograms_helper(Process * c_p, Eterm arg_tuple, int gather(Process *, int, int, int, UWord, Eterm)) { - SWord hist_start, hist_width, sched_id; + SWord hist_start, hist_width, aux_work_tid; int msg_count, alloc_num; Eterm *args; @@ -5147,15 +5147,15 @@ gather_histograms_helper(Process * c_p, Eterm arg_tuple, BIF_ERROR(c_p, BADARG); } - sched_id = signed_val(args[2]); + aux_work_tid = signed_val(args[2]); hist_width = signed_val(args[3]); hist_start = signed_val(args[4]); - if (sched_id < 0 || sched_id > erts_no_schedulers) { + if (aux_work_tid < 0 || erts_no_aux_work_threads <= aux_work_tid) { BIF_ERROR(c_p, BADARG); } - msg_count = gather(c_p, alloc_num, sched_id, hist_width, hist_start, args[5]); + msg_count = gather(c_p, alloc_num, aux_work_tid, hist_width, hist_start, args[5]); BIF_RET(make_small(msg_count)); } diff --git a/erts/emulator/beam/erl_bif_trace.c b/erts/emulator/beam/erl_bif_trace.c index 8024687c5e..e01ad49f79 100644 --- a/erts/emulator/beam/erl_bif_trace.c +++ b/erts/emulator/beam/erl_bif_trace.c @@ -2359,6 +2359,7 @@ trace_delivered_1(BIF_ALIST_1) (erts_aint32_t) erts_no_schedulers); erts_proc_add_refc(BIF_P, 1); erts_schedule_multi_misc_aux_work(0, + 1, erts_no_schedulers, reply_trace_delivered_all, (void *) tdarp); diff --git a/erts/emulator/beam/erl_bif_unique.h b/erts/emulator/beam/erl_bif_unique.h index 89d6e358fb..4c6335dc45 100644 --- a/erts/emulator/beam/erl_bif_unique.h +++ b/erts/emulator/beam/erl_bif_unique.h @@ -79,7 +79,7 @@ Eterm erts_debug_make_unique_integer(Process *c_p, Eterm etval1); -ERTS_GLB_INLINE void erts_set_ref_numbers(Uint32 *ref, Uint32 thr_id, Uint64 value); +ERTS_GLB_INLINE void erts_set_ref_numbers(Uint32 ref[ERTS_REF_NUMBERS], Uint32 thr_id, Uint64 value); ERTS_GLB_INLINE Uint32 erts_get_ref_numbers_thr_id(Uint32 *ref); ERTS_GLB_INLINE int erts_is_ref_numbers_magic(Uint32 *ref); ERTS_GLB_INLINE int erts_is_pid_ref_numbers(Uint32 *ref); diff --git a/erts/emulator/beam/erl_bits.c b/erts/emulator/beam/erl_bits.c index 2990f4e775..3cdbc66fb9 100644 --- a/erts/emulator/beam/erl_bits.c +++ b/erts/emulator/beam/erl_bits.c @@ -1419,7 +1419,7 @@ erts_bs_append(Process* c_p, Eterm* reg, Uint live, Eterm build_size_term, erts_bin_offset = 8*sb->size + sb->bitsize; if (unit > 1) { if ((unit == 8 && (erts_bin_offset & 7) != 0) || - (erts_bin_offset % unit) != 0) { + (unit != 8 && (erts_bin_offset % unit) != 0)) { goto badarg; } } @@ -1509,7 +1509,7 @@ erts_bs_append(Process* c_p, Eterm* reg, Uint live, Eterm build_size_term, erts_bin_offset = 8*binary_size(bin) + bitsize; if (unit > 1) { if ((unit == 8 && (erts_bin_offset & 7) != 0) || - (erts_bin_offset % unit) != 0) { + (unit != 8 && (erts_bin_offset % unit) != 0)) { goto badarg; } } diff --git a/erts/emulator/beam/erl_db.c b/erts/emulator/beam/erl_db.c index 06cf04f9ee..08fc410414 100644 --- a/erts/emulator/beam/erl_db.c +++ b/erts/emulator/beam/erl_db.c @@ -3197,12 +3197,21 @@ ets_all_reply(ErtsSchedulerData *esdp, ErtsEtsAllReq **reqpp, } int -erts_handle_yielded_ets_all_request(ErtsSchedulerData *esdp, - ErtsEtsAllYieldData *eaydp) +erts_handle_yielded_ets_all_request(ErtsAuxWorkData *awdp) { - int ix = (int) esdp->no - 1; - int yc = ERTS_ETS_ALL_TB_YCNT; + ErtsSchedulerData *esdp; + ErtsEtsAllYieldData *eaydp; + int ix, yc; + esdp = awdp->esdp; + /* only on normal scheduler threads... */ + if (!esdp || esdp->type != ERTS_SCHED_NORMAL) + return 0; + + eaydp = &awdp->yield.ets_all; + ix = (int) esdp->no - 1; + yc = ERTS_ETS_ALL_TB_YCNT; + while (1) { if (!eaydp->ongoing) { ErtsEtsAllReq *ongoing; @@ -3249,7 +3258,7 @@ handle_ets_all_request(void *vreq) eayp->ongoing = req; eayp->hfrag = hf; eayp->tab = tb; - erts_notify_new_aux_yield_work(esdp); + erts_more_yield_aux_work(&esdp->aux_work_data); } } else { @@ -3282,6 +3291,7 @@ BIF_RETTYPE ets_internal_request_all_0(BIF_ALIST_0) if (erts_no_schedulers > 1) erts_schedule_multi_misc_aux_work(1, + 1, erts_no_schedulers, handle_ets_all_request, (void *) req); @@ -5353,7 +5363,7 @@ static void lcnt_update_db_locks_per_sched(void *enable) { } void erts_lcnt_update_db_locks(int enable) { - erts_schedule_multi_misc_aux_work(0, erts_no_schedulers, + erts_schedule_multi_misc_aux_work(0, 1, erts_no_schedulers, &lcnt_update_db_locks_per_sched, (void*)(UWord)enable); } #endif /* ERTS_ENABLE_LOCK_COUNT */ diff --git a/erts/emulator/beam/erl_db.h b/erts/emulator/beam/erl_db.h index 2f6debe72a..a18be4d168 100644 --- a/erts/emulator/beam/erl_db.h +++ b/erts/emulator/beam/erl_db.h @@ -78,9 +78,7 @@ typedef struct { } DbTableRelease; struct ErtsSchedulerData_; -int erts_handle_yielded_ets_all_request(struct ErtsSchedulerData_ *esdp, - ErtsEtsAllYieldData *eadp); - +int erts_handle_yielded_ets_all_request(ErtsAuxWorkData *awdp); void erts_ets_sched_spec_data_init(struct ErtsSchedulerData_ *esdp); /* diff --git a/erts/emulator/beam/erl_gc.c b/erts/emulator/beam/erl_gc.c index 07a7b6c9e8..cf56d21f1e 100644 --- a/erts/emulator/beam/erl_gc.c +++ b/erts/emulator/beam/erl_gc.c @@ -3383,6 +3383,7 @@ erts_gc_info_request(Process *c_p) if (erts_no_schedulers > 1) erts_schedule_multi_misc_aux_work(1, + 1, erts_no_schedulers, reply_gc_info, (void *) gcirp); diff --git a/erts/emulator/beam/erl_init.c b/erts/emulator/beam/erl_init.c index cc2a168813..bb922cb412 100644 --- a/erts/emulator/beam/erl_init.c +++ b/erts/emulator/beam/erl_init.c @@ -643,7 +643,7 @@ void erts_usage(void) #ifdef BEAMASM erts_fprintf(stderr, "-JDdump bool enable or disable dumping of generated assembly code for each module loaded\n"); - erts_fprintf(stderr, "-JDperf bool enable or disable support for perf on Linux\n"); + erts_fprintf(stderr, "-JPperf bool enable or disable support for perf on Linux\n"); erts_fprintf(stderr, "\n"); #endif @@ -823,6 +823,7 @@ early_init(int *argc, char **argv) /* int dirty_cpu_scheds_pctg = 100; int dirty_cpu_scheds_onln_pctg = 100; int dirty_io_scheds; + int aux_threads = 1; int max_reader_groups; int reader_groups; int max_decentralized_counter_groups; @@ -1227,11 +1228,12 @@ early_init(int *argc, char **argv) /* erts_no_dirty_cpu_schedulers = no_dirty_cpu_schedulers = dirty_cpu_scheds; no_dirty_cpu_schedulers_online = dirty_cpu_scheds_online; erts_no_dirty_io_schedulers = no_dirty_io_schedulers = dirty_io_scheds; - erts_early_init_scheduling(no_schedulers); + erts_early_init_scheduling(no_schedulers + 1 + dirty_cpu_scheds); alloc_opts.ncpu = ncpu; erts_alloc_init(argc, argv, &alloc_opts); /* Handles (and removes) -M flags. */ + aux_threads += erts_no_dirty_alloc_instances; /* Require allocators */ erts_init_check_io(argc, argv); @@ -1241,7 +1243,7 @@ early_init(int *argc, char **argv) /* * * * Managed threads: * ** Scheduler threads (see erl_process.c) - * ** Aux thread (see erl_process.c) + * ** Aux threads (see erl_process.c) * ** Sys message dispatcher thread (see erl_trace.c) * ** IO Poll threads (see erl_check_io.c) * @@ -1250,11 +1252,13 @@ early_init(int *argc, char **argv) /* * ** Dirty scheduler threads */ erts_thr_progress_init(no_schedulers, - no_schedulers+2+erts_no_poll_threads, - erts_async_max_threads + - erts_no_dirty_cpu_schedulers + - erts_no_dirty_io_schedulers - ); + (no_schedulers + + aux_threads + + 1 + + erts_no_poll_threads), + (erts_async_max_threads + + erts_no_dirty_cpu_schedulers + + erts_no_dirty_io_schedulers)); erts_thr_q_init(); erts_init_utils(); erts_early_init_cpu_topology(no_schedulers, diff --git a/erts/emulator/beam/erl_message.c b/erts/emulator/beam/erl_message.c index a6eecfabdc..4b7bb22a31 100644 --- a/erts/emulator/beam/erl_message.c +++ b/erts/emulator/beam/erl_message.c @@ -1072,6 +1072,7 @@ void erts_factory_proc_init(ErtsHeapFactory* factory, Process* p) factory->mode = FACTORY_HALLOC; factory->p = p; factory->hp_start = HEAP_TOP(p); + factory->original_htop = factory->hp_start; factory->hp = factory->hp_start; if (factory->hp) factory->hp_end = HEAP_LIMIT(p); @@ -1097,6 +1098,11 @@ void erts_factory_proc_prealloc_init(ErtsHeapFactory* factory, ErlHeapFragment *bp = p->mbuf; factory->mode = FACTORY_HALLOC; factory->p = p; + factory->original_htop = HEAP_TOP(p); + /* + factory->hp_start is a pointer to somewhere in the data area of + a heap fragment or to the main heap. + */ factory->hp_start = HAlloc(p, size); factory->hp = factory->hp_start; factory->hp_end = factory->hp_start + size; @@ -1163,6 +1169,12 @@ erts_factory_message_create(ErtsHeapFactory* factory, ASSERT(ohp == &proc->off_heap); factory->mode = FACTORY_HALLOC; factory->p = proc; + /* + If on_heap is set then hp must be on the process main heap. + */ + factory->original_htop = hp; + ASSERT(HEAP_START(proc) <= factory->original_htop); + ASSERT(factory->original_htop <= HEAP_LIMIT(proc)); factory->heap_frags_saved = proc->mbuf; factory->heap_frags_saved_used = proc->mbuf ? proc->mbuf->used_size : 0; } @@ -1495,10 +1507,10 @@ void erts_factory_undo(ErtsHeapFactory* factory) /* Rollback heap top */ - if (HEAP_START(factory->p) <= factory->hp_start - && factory->hp_start <= HEAP_LIMIT(factory->p)) { - HEAP_TOP(factory->p) = factory->hp_start; - } + ASSERT(HEAP_START(factory->p) <= factory->original_htop); + ASSERT(factory->original_htop <= HEAP_LIMIT(factory->p)); + HEAP_TOP(factory->p) = factory->original_htop; + /* Fix last heap frag */ if (factory->heap_frags_saved) { diff --git a/erts/emulator/beam/erl_message.h b/erts/emulator/beam/erl_message.h index 89017a3998..85a16cb605 100644 --- a/erts/emulator/beam/erl_message.h +++ b/erts/emulator/beam/erl_message.h @@ -74,7 +74,27 @@ typedef struct { FACTORY_TMP } mode; Process* p; + /* + If the factory is initialized with erts_factory_proc_prealloc_init, + hp_start points to the top of the main heap if the preallocated data + fits in the main heap and otherwise it points to somewhere in the + data area of a heap fragment. If the factory is initialized with any + of the other init functions that sets the mode to FACTORY_HALLOC, + hp_start and original_htop always have the same value. + + When erts_factory_proc_prealloc_init is used for initialization the + preallocated data might be allocated in an existing heap fragment but + data that is later allocated with erts_produce_heap might fit in the + main heap, so both hp_start and original_htop are needed to correctly + restore the heap in the erts_factory_undo function. + */ Eterm* hp_start; + /* + original_htop stores the top of the main heap at the time + the factory was initialized and is used to reset the heap + state if an erts_factory_undo call is made. + */ + Eterm* original_htop; Eterm* hp; Eterm* hp_end; ErtsMessage *message; diff --git a/erts/emulator/beam/erl_msacc.c b/erts/emulator/beam/erl_msacc.c index ed47cb0b83..89fdaf77cd 100644 --- a/erts/emulator/beam/erl_msacc.c +++ b/erts/emulator/beam/erl_msacc.c @@ -364,20 +364,17 @@ erts_msacc_request(Process *c_p, int action, Eterm *threads) msaccrp->ref = STORE_NC(&hp, NULL, ref); msaccrp->req_sched = esdp->no; - *threads = erts_no_schedulers; - *threads += 1; /* aux thread */ + *threads = erts_no_aux_work_threads; erts_atomic32_init_nob(&msaccrp->refc,(erts_aint32_t)*threads); erts_proc_add_refc(c_p, *threads); - if (erts_no_schedulers > 1) - erts_schedule_multi_misc_aux_work(1, - erts_no_schedulers, - reply_msacc, - (void *) msaccrp); - /* aux thread */ - erts_schedule_misc_aux_work(0, reply_msacc, (void *) msaccrp); + erts_schedule_multi_misc_aux_work(1, + 0, + erts_no_aux_work_threads-1, + reply_msacc, + (void *) msaccrp); /* Manage unmanaged threads */ switch (action) { diff --git a/erts/emulator/beam/erl_proc_sig_queue.c b/erts/emulator/beam/erl_proc_sig_queue.c index aabd9f7aa6..46bdbb277d 100644 --- a/erts/emulator/beam/erl_proc_sig_queue.c +++ b/erts/emulator/beam/erl_proc_sig_queue.c @@ -252,6 +252,7 @@ handle_move_msgq_off_heap(Process *c_p, static void send_cla_reply(Process *c_p, ErtsMessage *sig, Eterm to, Eterm req_id, Eterm result); +static void handle_missing_spawn_reply(Process *c_p, ErtsMonitor *omon); #ifdef ERTS_PROC_SIG_HARD_DEBUG #define ERTS_PROC_SIG_HDBG_PRIV_CHKQ(P, T, NMN) \ @@ -1927,7 +1928,7 @@ reply_dist_unlink_ack(Process *c_p, ErtsSigDistUnlinkOp *sdulnk) */ if (dep != erts_this_dist_entry && sdulnk->nodename == dep->sysname) { ErtsDSigSendContext ctx; - int code = erts_dsig_prepare(&ctx, dep, c_p, 0, + int code = erts_dsig_prepare(&ctx, dep, NULL, 0, ERTS_DSP_NO_LOCK, 1, 1, 0); switch (code) { case ERTS_DSIG_PREP_CONNECTED: @@ -4619,14 +4620,14 @@ handle_dist_spawn_reply(Process *c_p, ErtsSigRecvTracing *tracing, if (dep != erts_this_dist_entry && dist->nodename == dep->sysname) { ErtsDSigSendContext ctx; - int code = erts_dsig_prepare(&ctx, dep, c_p, 0, + int code = erts_dsig_prepare(&ctx, dep, NULL, 0, ERTS_DSP_NO_LOCK, 1, 1, 0); switch (code) { case ERTS_DSIG_PREP_CONNECTED: case ERTS_DSIG_PREP_PENDING: if (dist->connection_id == ctx.connection_id) { code = erts_dsig_send_exit_tt(&ctx, - c_p->common.id, + c_p, result, am_abandoned, SEQ_TRACE_TOKEN(c_p)); @@ -5076,11 +5077,27 @@ erts_proc_sig_handle_incoming(Process *c_p, erts_aint32_t *statep, xsigd->u.ref); if (omon) { ASSERT(erts_monitor_is_origin(omon)); + if (omon->type == ERTS_MON_TYPE_ALIAS) { + omon = NULL; + break; + } + if (omon->flags & ERTS_ML_FLG_SPAWN_PENDING) { + handle_missing_spawn_reply(c_p, omon); + /* + * We leave the pending spawn monitor as is, + * so that the nodedown will trigger an error + * spawn_reply... + */ + omon = NULL; + cnt += 4; + break; + } mdp = erts_monitor_to_data(omon); if (omon->type == ERTS_MON_TYPE_DIST_PROC) { if (erts_monitor_dist_delete(&mdp->u.target)) tmon = &mdp->u.target; } + ASSERT(!(omon->flags & ERTS_ML_FLGS_SPAWN)); cnt += convert_prepared_down_message(c_p, sig, xsigd->message, next_nm_sig); @@ -6981,6 +6998,42 @@ handle_msg_tracing(Process *c_p, ErtsSigRecvTracing *tracing, return 0; } +static void +handle_missing_spawn_reply(Process *c_p, ErtsMonitor *omon) +{ + ErtsMonitorData *mdp; + ErtsMonitorDataExtended *mdep; + erts_dsprintf_buf_t *dsbufp; + Eterm nodename; + DistEntry *dep; + + /* Terminate connection to the node and report it... */ + + if (omon->type != ERTS_MON_TYPE_DIST_PROC) + ERTS_INTERNAL_ERROR("non-distributed missing spawn_reply"); + + mdp = erts_monitor_to_data(omon); + ASSERT(mdp->origin.flags & ERTS_ML_FLG_EXTENDED); + mdep = (ErtsMonitorDataExtended *) mdp; + ASSERT(mdep->dist); + nodename = mdep->dist->nodename; + ASSERT(is_atom(nodename)); + + dep = erts_find_dist_entry(nodename); + if (dep) + erts_kill_dist_connection(dep, mdep->dist->connection_id); + + dsbufp = erts_create_logger_dsbuf(); + erts_dsprintf(dsbufp, + "Missing 'spawn_reply' signal from the node %T " + "detected by %T on the node %T. The node %T " + "probably suffers from the bug with ticket id " + "OTP-17737.", + nodename, c_p->common.id, + erts_this_dist_entry->sysname, nodename); + erts_send_error_to_logger_nogl(dsbufp); +} + Uint erts_proc_sig_prep_msgq_for_inspection(Process *c_p, Process *rp, diff --git a/erts/emulator/beam/erl_proc_sig_queue.h b/erts/emulator/beam/erl_proc_sig_queue.h index 236b20ab4d..90a87570c2 100644 --- a/erts/emulator/beam/erl_proc_sig_queue.h +++ b/erts/emulator/beam/erl_proc_sig_queue.h @@ -1457,6 +1457,22 @@ ERTS_GLB_INLINE void erts_msgq_set_save_first(Process *c_p); /** * + * @brief Remove a message from the message queue and set + * the save pointer to the start of the message queue. + * + * + * @param[in] c_p Pointer to process struct of + * currently executing process. + * + * @param[in] msgp A pointer to the message to + * remove from the message queue. + * + */ +ERTS_GLB_INLINE void erts_msgq_unlink_msg_set_save_first(Process *c_p, + ErtsMessage *msgp); + +/** + * * @brief Advance the save pointer to the next message in the * message queue. * @@ -1865,6 +1881,21 @@ erts_msgq_set_save_first(Process *c_p) } ERTS_GLB_INLINE void +erts_msgq_unlink_msg_set_save_first(Process *c_p, ErtsMessage *msgp) +{ + ErtsMessage *sigp = msgp->next; + ERTS_HDBG_CHECK_SIGNAL_PRIV_QUEUE__(c_p, 0, "before"); + *c_p->sig_qs.save = sigp; + c_p->sig_qs.len--; + if (!sigp) + c_p->sig_qs.last = c_p->sig_qs.save; + else if (ERTS_SIG_IS_RECV_MARKER(sigp)) + ((ErtsRecvMarker *) sigp)->prev_next = c_p->sig_qs.save; + erts_msgq_set_save_first(c_p); + ERTS_HDBG_CHECK_SIGNAL_PRIV_QUEUE__(c_p, 0, "after"); +} + +ERTS_GLB_INLINE void erts_msgq_set_save_next(Process *c_p) { ErtsMessage *sigp = (*c_p->sig_qs.save)->next; diff --git a/erts/emulator/beam/erl_process.c b/erts/emulator/beam/erl_process.c index eacbb90770..8313a4863e 100644 --- a/erts/emulator/beam/erl_process.c +++ b/erts/emulator/beam/erl_process.c @@ -137,6 +137,7 @@ Uint ERTS_WRITE_UNLIKELY(erts_no_schedulers); Uint ERTS_WRITE_UNLIKELY(erts_no_total_schedulers); Uint ERTS_WRITE_UNLIKELY(erts_no_dirty_cpu_schedulers) = 0; Uint ERTS_WRITE_UNLIKELY(erts_no_dirty_io_schedulers) = 0; +int ERTS_WRITE_UNLIKELY(erts_no_aux_work_threads); static char *erts_aux_work_flag_descr[ERTS_SSI_AUX_WORK_NO_FLAGS] = {0}; int erts_aux_work_no_flags = ERTS_SSI_AUX_WORK_NO_FLAGS; @@ -171,7 +172,12 @@ sched_get_busy_wait_params(ErtsSchedulerData *esdp) return &sched_busy_wait_params[esdp->type]; } -static ErtsAuxWorkData *aux_thread_aux_work_data; +typedef union { + ErtsAuxWorkData data; + char align[ERTS_ALC_CACHE_LINE_ALIGN_SIZE(sizeof(ErtsAuxWorkData))]; +} ErtsAlignedAuxWorkData; + +static ErtsAlignedAuxWorkData *ERTS_WRITE_UNLIKELY(aligned_aux_work_data); #define ERTS_SCHDLR_SSPND_CHNG_NMSB (((erts_aint32_t) 1) << 0) #define ERTS_SCHDLR_SSPND_CHNG_MSB (((erts_aint32_t) 1) << 1) @@ -485,8 +491,8 @@ ERTS_SCHED_PREF_QUICK_ALLOC_IMPL(proclist, ERTS_ALC_T_PROC_LIST) #define ERTS_SCHED_SLEEP_INFO_IX(IX) \ - (ASSERT(((int)-1) <= ((int) (IX)) \ - && ((int) (IX)) < ((int) erts_no_schedulers)), \ + (ASSERT((((int)-1) <= ((int) (IX))) \ + && (((int) (IX)) < (erts_no_aux_work_threads-1))), \ &aligned_sched_sleep_info[(IX)].ssi) #define ERTS_DIRTY_CPU_SCHED_SLEEP_INFO_IX(IX) \ (ASSERT(0 <= ((int) (IX)) \ @@ -555,7 +561,7 @@ static void print_function_from_pc(fmtfn_t to, void *to_arg, ErtsCodePtr x); static int stack_element_dump(fmtfn_t to, void *to_arg, Eterm* sp, int yreg); static void aux_work_timeout(void *unused); -static void aux_work_timeout_early_init(int no_schedulers); +static void aux_work_timeout_early_init(int max_no_aux_work_threads); static void setup_aux_work_timer(ErtsSchedulerData *esdp); static int execute_sys_tasks(Process *c_p, @@ -1393,6 +1399,7 @@ erts_sched_wall_time_request(Process *c_p, int set, int enable, if (erts_no_schedulers > 1) erts_schedule_multi_misc_aux_work(1, + 1, erts_no_schedulers, reply_sched_wall_time, (void *) swtrp); @@ -1457,6 +1464,7 @@ Eterm erts_system_check_request(Process *c_p) { if (erts_no_schedulers > 1) erts_schedule_multi_misc_aux_work(1, + 1, erts_no_schedulers, reply_system_check, (void *) scrp); @@ -1814,11 +1822,9 @@ init_misc_aux_work(void) misc_aux_work_queues = erts_alloc_permanent_cache_aligned(ERTS_ALC_T_MISC_AUX_WORK_Q, sizeof(erts_algnd_misc_aux_work_q_t) - * (erts_no_schedulers+1)); - - ix = 0; /* aux_thread + schedulers */ + * erts_no_aux_work_threads); - for (; ix <= erts_no_schedulers; ix++) { + for (ix = 0; ix < erts_no_aux_work_threads; ix++) { qinit.arg = (void *) ERTS_SCHED_SLEEP_INFO_IX(ix-1); erts_thr_q_initialize(&misc_aux_work_queues[ix].q, &qinit); } @@ -1849,7 +1855,7 @@ handle_misc_aux_work(ErtsAuxWorkData *awdp, erts_aint32_t aux_work, int waiting) { - ErtsThrQ_t *q = &misc_aux_work_queues[awdp->sched_id].q; + ErtsThrQ_t *q = &misc_aux_work_queues[awdp->aux_work_tid].q; unset_aux_work_flags_mb(awdp->ssi, ERTS_SSI_AUX_WORK_MISC); while (1) { @@ -1877,23 +1883,36 @@ handle_misc_aux_work_thr_prgr(ErtsAuxWorkData *awdp, unset_aux_work_flags(awdp->ssi, ERTS_SSI_AUX_WORK_MISC_THR_PRGR); - return misc_aux_work_clean(&misc_aux_work_queues[awdp->sched_id].q, + return misc_aux_work_clean(&misc_aux_work_queues[awdp->aux_work_tid].q, awdp, aux_work & ~ERTS_SSI_AUX_WORK_MISC_THR_PRGR); } +static ERTS_INLINE int +get_aux_work_tid(void) +{ + /* + * All aux work threads are registered as delayed dealloc + * handlers, and their allocation index equals their tid. + */ + ErtsThrAllocData *tadp = erts_get_thr_alloc_data(); + if (!tadp || !tadp->delayed_dealloc_handler) + return -1; /* Not an aux work thread */ + return tadp->alc_ix; +} static ERTS_INLINE void -schedule_misc_aux_work(int sched_id, +schedule_misc_aux_work(int aux_work_tid, void (*func)(void *), void *arg) { ErtsThrQ_t *q; erts_misc_aux_work_t *mawp; - ASSERT(0 <= sched_id && sched_id <= erts_no_schedulers); + ASSERT(0 <= aux_work_tid + && aux_work_tid < erts_no_aux_work_threads); - q = &misc_aux_work_queues[sched_id].q; + q = &misc_aux_work_queues[aux_work_tid].q; mawp = misc_aux_work_alloc(); mawp->func = func; mawp->arg = arg; @@ -1901,37 +1920,53 @@ schedule_misc_aux_work(int sched_id, } void -erts_schedule_misc_aux_work(int sched_id, +erts_schedule_misc_aux_work(int aux_work_tid, void (*func)(void *), void *arg) { - schedule_misc_aux_work(sched_id, func, arg); + schedule_misc_aux_work(aux_work_tid, func, arg); } void erts_schedule_multi_misc_aux_work(int ignore_self, - int max_sched, + int min_tid, + int max_tid, void (*func)(void *), void *arg) { - int id, self = 0; - - if (ignore_self) { - ErtsSchedulerData *esdp = erts_get_scheduler_data(); + int tid, self; - /* ignore_self is meaningless on dirty schedulers since aux work can - * only run on normal schedulers, and their ids do not translate. */ - if(esdp && !ERTS_SCHEDULER_IS_DIRTY(esdp)) { - self = (int)esdp->no; - } - } + /* + * Threads handling misc aux work are: + * * Normal scheduler threads + * * Aux work threads + * + * Tids corresponds to threads as follows: + * * tid == 0 + * Standard aux thread. This thread is always + * present on all systems. + * * 1 <= tid <= erts_no_schedulers + * Normal scheduler threads. There are always at + * least one normal scheduler. Tid equals scheduler + * id. + * * erts_no_schedulers < tid < erts_no_aux_work_threads + * Extra aux threads. Main purpose to handle + * delayed dealloc for allocator instances for + * dirty schedulers. May or may not exist. Maximum + * amount of these threads equals amount of dirty + * cpu schedulers. + */ - ASSERT(0 < max_sched && max_sched <= erts_no_schedulers); + ASSERT(0 <= min_tid && min_tid < erts_no_aux_work_threads); + ASSERT(0 <= max_tid && max_tid < erts_no_aux_work_threads); + ASSERT(min_tid <= max_tid); - for (id = 1; id <= max_sched; id++) { - if (id == self) + self = ignore_self ? get_aux_work_tid() : -1; + + for (tid = min_tid; tid <= max_tid; tid++) { + if (tid == self) continue; - schedule_misc_aux_work(id, func, arg); + schedule_misc_aux_work(tid, func, arg); } } @@ -2013,7 +2048,7 @@ handle_fix_alloc(ErtsAuxWorkData *awdp, erts_aint32_t aux_work, int waiting) | ERTS_SSI_AUX_WORK_FIX_ALLOC_DEALLOC)); aux_work &= ~(ERTS_SSI_AUX_WORK_FIX_ALLOC_LOWER_LIM | ERTS_SSI_AUX_WORK_FIX_ALLOC_DEALLOC); - res = erts_alloc_fix_alloc_shrink(awdp->sched_id, aux_work); + res = erts_alloc_fix_alloc_shrink(awdp->aux_work_tid, aux_work); if (res) { set_aux_work_flags(ssi, res); aux_work |= res; @@ -2060,10 +2095,10 @@ handle_delayed_dealloc(ErtsAuxWorkData *awdp, erts_aint32_t aux_work, int waitin unset_aux_work_flags_mb(ssi, ERTS_SSI_AUX_WORK_DD); ERTS_MSACC_SET_STATE_CACHED_M_X(ERTS_MSACC_STATE_ALLOC); - erts_alloc_scheduler_handle_delayed_dealloc((void *) awdp->esdp, - &need_thr_progress, - &wakeup, - &more_work); + erts_alloc_handle_delayed_dealloc(&awdp->alloc_data, + &need_thr_progress, + &wakeup, + &more_work); ERTS_MSACC_POP_STATE_M_X(); if (more_work) { if (set_aux_work_flags(ssi, ERTS_SSI_AUX_WORK_DD) @@ -2103,10 +2138,10 @@ handle_delayed_dealloc_thr_prgr(ErtsAuxWorkData *awdp, erts_aint32_t aux_work, i need_thr_progress = 0; more_work = 0; - erts_alloc_scheduler_handle_delayed_dealloc((void *) awdp->esdp, - &need_thr_progress, - &wakeup, - &more_work); + erts_alloc_handle_delayed_dealloc(&awdp->alloc_data, + &need_thr_progress, + &wakeup, + &more_work); if (more_work) { set_aux_work_flags(ssi, ERTS_SSI_AUX_WORK_DD); unset_aux_work_flags(ssi, ERTS_SSI_AUX_WORK_DD_THR_PRGR); @@ -2347,13 +2382,24 @@ setup_thr_debug_wait_completed(void *vproc) ErtsSchedulerData *esdp = erts_get_scheduler_data(); ErtsAuxWorkData *awdp; erts_aint32_t wait_flags, aux_work_flags; - awdp = esdp ? &esdp->aux_work_data : aux_thread_aux_work_data; + + if (esdp) + awdp = &esdp->aux_work_data; /* A normal scheduler... */ + else { + /* An aux thread... */ + ErtsThrAllocData *tadp = erts_get_thr_alloc_data(); + char *ptr; + ASSERT(tadp); + ptr = (char *) tadp; + ptr -= offsetof(ErtsAuxWorkData, alloc_data); + awdp = (ErtsAuxWorkData *) ptr; + } wait_flags = 0; aux_work_flags = ERTS_SSI_AUX_WORK_DEBUG_WAIT_COMPLETED; if (debug_wait_completed_flags & ERTS_DEBUG_WAIT_COMPLETED_DEALLOCATIONS) { - erts_alloc_fix_alloc_shrink(awdp->sched_id, 0); + erts_alloc_fix_alloc_shrink(awdp->aux_work_tid, 0); wait_flags |= (ERTS_SSI_AUX_WORK_DD | ERTS_SSI_AUX_WORK_DD_THR_PRGR); aux_work_flags |= ERTS_SSI_AUX_WORK_DD; @@ -2383,19 +2429,15 @@ static void later_thr_debug_wait_completed(void *vlop) struct debug_lop *lop = vlop; if (erts_atomic32_dec_read_mb(&debug_wait_completed_count) == 1) { - erts_aint32_t count = (erts_aint32_t) erts_no_schedulers; - count += 1; /* aux thread */ + erts_aint32_t count = (erts_aint32_t) erts_no_aux_work_threads; erts_atomic32_set_nob(&debug_wait_completed_count, count); - /* scheduler threads */ + /* All scheduler threads as well as all aux threads... */ erts_schedule_multi_misc_aux_work(0, - erts_no_schedulers, + 0, + erts_no_aux_work_threads-1, setup_thr_debug_wait_completed, lop->proc); - /* aux_thread */ - erts_schedule_misc_aux_work(0, - setup_thr_debug_wait_completed, - lop->proc); } erts_free(ERTS_ALC_T_DEBUG, lop); } @@ -2435,6 +2477,7 @@ erts_debug_wait_completed(Process *c_p, int flags) /* First flush later-ops on all scheduler threads */ erts_schedule_multi_misc_aux_work(0, + 1, erts_no_schedulers, init_thr_debug_wait_completed, (void *) c_p); @@ -2507,11 +2550,27 @@ handle_reap_ports(ErtsAuxWorkData *awdp, erts_aint32_t aux_work, int waiting) } void -erts_notify_new_aux_yield_work(ErtsSchedulerData *esdp) +erts_more_yield_aux_work(ErtsAuxWorkData *awdp) { - ASSERT(esdp == erts_get_scheduler_data()); - /* Always called by the scheduler itself... */ - set_aux_work_flags_wakeup_nob(esdp->ssi, ERTS_SSI_AUX_WORK_YIELD); + /* Should *always* be called by the aux-work thread itself... */ + ASSERT(awdp && awdp->aux_work_tid == get_aux_work_tid()); + set_aux_work_flags_wakeup_nob(awdp->ssi, ERTS_SSI_AUX_WORK_YIELD); +} + +ErtsAuxWorkData * +erts_get_aux_work_data(void) +{ + ErtsAuxWorkData *awdp; + int tid = get_aux_work_tid(); + ASSERT(0 <= tid && tid < erts_no_aux_work_threads); + if (tid < 1) + awdp = tid == 0 ? &aligned_aux_work_data[0].data : NULL; + else if (tid <= erts_no_schedulers) + awdp = &erts_aligned_scheduler_data[tid-1].esd.aux_work_data; + else + awdp = &aligned_aux_work_data[tid - (int) erts_no_schedulers].data; + ASSERT(!awdp || awdp->aux_work_tid == tid); + return awdp; } static ERTS_INLINE erts_aint32_t @@ -2531,10 +2590,8 @@ handle_yield(ErtsAuxWorkData *awdp, erts_aint32_t aux_work, int waiting) /* Various yielding operations... */ - yield |= erts_handle_yielded_ets_all_request(awdp->esdp, - &awdp->yield.ets_all); - yield |= erts_handle_yielded_alcu_blockscan(awdp->esdp, - &awdp->yield.alcu_blockscan); + yield |= erts_handle_yielded_ets_all_request(awdp); + yield |= erts_handle_yielded_alcu_blockscan(awdp); /* * Other yielding operations... @@ -2718,7 +2775,7 @@ start_aux_work_timer(ErtsSchedulerData *esdp) } static void -aux_work_timeout_early_init(int no_schedulers) +aux_work_timeout_early_init(int max_no_aux_work_threads) { int i; UWord p; @@ -2729,7 +2786,8 @@ aux_work_timeout_early_init(int no_schedulers) */ p = (UWord) malloc((sizeof(ErtsAuxWorkTmo) - + sizeof(erts_atomic32_t)*(no_schedulers+1)) + + (sizeof(erts_atomic32_t) + * max_no_aux_work_threads)) + ERTS_CACHE_LINE_SIZE-1); if (!p) { ERTS_INTERNAL_ERROR("malloc failed to allocate memory!"); @@ -2744,7 +2802,7 @@ aux_work_timeout_early_init(int no_schedulers) #ifdef DEBUG erts_atomic32_init_nob(&aux_work_tmo->used, 0); #endif - for (i = 0; i <= no_schedulers; i++) + for (i = 0; i < max_no_aux_work_threads; i++) erts_atomic32_init_nob(&aux_work_tmo->type[i], 0); } @@ -3079,37 +3137,49 @@ thr_prgr_fin_wait(void *vssi) | ERTS_SSI_FLG_TSE_SLEEPING)); } -static void init_aux_work_data(ErtsAuxWorkData *awdp, ErtsSchedulerData *esdp, char *dawwp); +static void init_aux_work_data(ErtsAuxWorkData *awdp, ErtsSchedulerData *esdp, + char *dawwp); void -erts_aux_thread_poke() +erts_aux_thread_poke(void) { erts_sched_poke(ERTS_SCHED_SLEEP_INFO_IX(-1)); } static void * -aux_thread(void *unused) +aux_thread(void *vix) { - ErtsAuxWorkData *awdp = aux_thread_aux_work_data; - ErtsSchedulerSleepInfo *ssi = ERTS_SCHED_SLEEP_INFO_IX(-1); + int ix = (int) (Sint) vix; + int id = ix == 0 ? 1 : ix + 1 - erts_no_schedulers; + ErtsAuxWorkData *awdp = &aligned_aux_work_data[id-1].data; + ErtsSchedulerSleepInfo *ssi = ERTS_SCHED_SLEEP_INFO_IX(ix-1); erts_aint32_t aux_work; ErtsThrPrgrCallbacks callbacks; ErtsThrPrgrData *tpd; int thr_prgr_active = 1; ERTS_MSACC_DECLARE_CACHE(); + ASSERT(ix == 0 + || (erts_no_schedulers < ix + && ix < erts_no_aux_work_threads)); + ASSERT(0 < id && id <= 1 + erts_no_dirty_alloc_instances); + #ifdef ERTS_ENABLE_LOCK_CHECK { - char buf[] = "aux_thread"; - erts_lc_set_thread_name(buf); + char buf[31]; + erts_snprintf(&buf[0], 31, "aux-thread %d", id); + erts_lc_set_thread_name(&buf[0]); } #endif - erts_port_task_pre_alloc_init_thread(); + if (ix == 0) + erts_port_task_pre_alloc_init_thread(); ssi->event = erts_tse_fetch(); erts_tse_return(ssi->event); - erts_msacc_init_thread("aux", 1, 1); + erts_msacc_init_thread("aux", id, 1); + + erts_alloc_register_delayed_dealloc_handler_thread(&awdp->alloc_data, ix); callbacks.arg = (void *) ssi; callbacks.wakeup = thr_prgr_wakeup; @@ -3120,13 +3190,16 @@ aux_thread(void *unused) tpd = erts_thr_progress_register_managed_thread(NULL, &callbacks, 1, 0); init_aux_work_data(awdp, NULL, NULL); awdp->ssi = ssi; + awdp->aux_work_tid = ix; #if ERTS_POLL_USE_FALLBACK + if (ix == 0) { #if ERTS_POLL_USE_SCHEDULER_POLLING - ssi->psi = erts_create_pollset_thread(-2, tpd); + ssi->psi = erts_create_pollset_thread(-2, tpd); #else - ssi->psi = erts_create_pollset_thread(-1, tpd); + ssi->psi = erts_create_pollset_thread(-1, tpd); #endif + } #endif sched_prep_spin_wait(ssi); @@ -3149,7 +3222,7 @@ aux_thread(void *unused) if (!aux_work) { #ifdef ERTS_BREAK_REQUESTED - if (ERTS_BREAK_REQUESTED) + if (ix == 0 && ERTS_BREAK_REQUESTED) erts_do_break_handling(); #endif @@ -3157,40 +3230,43 @@ aux_thread(void *unused) erts_thr_progress_active(tpd, thr_prgr_active = 0); #if ERTS_POLL_USE_FALLBACK + if (ix == 0) { - flgs = sched_spin_wait(ssi, 0); + flgs = sched_spin_wait(ssi, 0); - if (flgs & ERTS_SSI_FLG_SLEEPING) { - ASSERT(flgs & ERTS_SSI_FLG_WAITING); - flgs = sched_set_sleeptype(ssi, ERTS_SSI_FLG_POLL_SLEEPING); - if (flgs & ERTS_SSI_FLG_SLEEPING) { - ASSERT(flgs & ERTS_SSI_FLG_POLL_SLEEPING); - ASSERT(flgs & ERTS_SSI_FLG_WAITING); - erts_check_io(ssi->psi, ERTS_POLL_INF_TIMEOUT, 0); - } + if (flgs & ERTS_SSI_FLG_SLEEPING) { + ASSERT(flgs & ERTS_SSI_FLG_WAITING); + flgs = sched_set_sleeptype(ssi, ERTS_SSI_FLG_POLL_SLEEPING); + if (flgs & ERTS_SSI_FLG_SLEEPING) { + ASSERT(flgs & ERTS_SSI_FLG_POLL_SLEEPING); + ASSERT(flgs & ERTS_SSI_FLG_WAITING); + erts_check_io(ssi->psi, ERTS_POLL_INF_TIMEOUT, 0); + } + } } -#else - erts_thr_progress_prepare_wait(tpd); - - flgs = sched_spin_wait(ssi, 0); + else +#endif + { + erts_thr_progress_prepare_wait(tpd); + flgs = sched_spin_wait(ssi, 0); - if (flgs & ERTS_SSI_FLG_SLEEPING) { - flgs = sched_set_sleeptype(ssi, ERTS_SSI_FLG_TSE_SLEEPING); - if (flgs & ERTS_SSI_FLG_SLEEPING) { - int res; - ASSERT(flgs & ERTS_SSI_FLG_TSE_SLEEPING); - ASSERT(flgs & ERTS_SSI_FLG_WAITING); - ERTS_MSACC_SET_STATE_CACHED(ERTS_MSACC_STATE_SLEEP); - do { - res = erts_tse_wait(ssi->event); - } while (res == EINTR); - ERTS_MSACC_SET_STATE_CACHED(ERTS_MSACC_STATE_OTHER); - } - erts_tse_return(ssi->event); + if (flgs & ERTS_SSI_FLG_SLEEPING) { + flgs = sched_set_sleeptype(ssi, ERTS_SSI_FLG_TSE_SLEEPING); + if (flgs & ERTS_SSI_FLG_SLEEPING) { + int res; + ASSERT(flgs & ERTS_SSI_FLG_TSE_SLEEPING); + ASSERT(flgs & ERTS_SSI_FLG_WAITING); + ERTS_MSACC_SET_STATE_CACHED(ERTS_MSACC_STATE_SLEEP); + do { + res = erts_tse_wait(ssi->event); + } while (res == EINTR); + ERTS_MSACC_SET_STATE_CACHED(ERTS_MSACC_STATE_OTHER); + } + erts_tse_return(ssi->event); + } + erts_thr_progress_finalize_wait(tpd); } - erts_thr_progress_finalize_wait(tpd); -#endif - } + } flgs = sched_prep_spin_wait(ssi); } @@ -5609,11 +5685,11 @@ runq_supervisor(void *unused) void -erts_early_init_scheduling(int no_schedulers) +erts_early_init_scheduling(int max_no_aux_work_threads) { ErtsSchedType type; - aux_work_timeout_early_init(no_schedulers); + aux_work_timeout_early_init(max_no_aux_work_threads); for (type = ERTS_SCHED_TYPE_FIRST; type <= ERTS_SCHED_TYPE_LAST; type++) { erts_sched_set_wakeup_other_threshold(type, "medium"); @@ -5723,7 +5799,8 @@ erts_sched_set_wake_cleanup_threshold(char *str) } static void -init_aux_work_data(ErtsAuxWorkData *awdp, ErtsSchedulerData *esdp, char *dawwp) +init_aux_work_data(ErtsAuxWorkData *awdp, ErtsSchedulerData *esdp, + char *dawwp) { int id = 0; if (esdp) { @@ -5746,7 +5823,7 @@ init_aux_work_data(ErtsAuxWorkData *awdp, ErtsSchedulerData *esdp, char *dawwp) } } - awdp->sched_id = id; + awdp->aux_work_tid = id; awdp->esdp = esdp; awdp->ssi = esdp ? esdp->ssi : NULL; awdp->latest_wakeup = ERTS_THR_PRGR_VAL_FIRST; @@ -5769,12 +5846,13 @@ init_aux_work_data(ErtsAuxWorkData *awdp, ErtsSchedulerData *esdp, char *dawwp) else { int i; awdp->delayed_wakeup.job = (ErtsDelayedAuxWorkWakeupJob *) dawwp; - dawwp += sizeof(ErtsDelayedAuxWorkWakeupJob)*(erts_no_schedulers+1); + dawwp += sizeof(ErtsDelayedAuxWorkWakeupJob)*erts_no_aux_work_threads; awdp->delayed_wakeup.sched2jix = (int *) dawwp; awdp->delayed_wakeup.jix = -1; - for (i = 0; i <= erts_no_schedulers; i++) + for (i = 0; i < erts_no_aux_work_threads; i++) awdp->delayed_wakeup.sched2jix[i] = -1; } + erts_alcu_blockscan_init(awdp); awdp->debug.wait_completed.flags = 0; awdp->debug.wait_completed.callback = NULL; awdp->debug.wait_completed.arg = NULL; @@ -5896,7 +5974,7 @@ erts_init_scheduling(int no_schedulers, int no_schedulers_online, int no_poll_th int no_dirty_io_schedulers ) { - int ix, n, no_ssi, tot_rqs; + int ix, n, tot_rqs; char *daww_ptr; size_t daww_sz; size_t size_runqs; @@ -6019,13 +6097,19 @@ erts_init_scheduling(int no_schedulers, int no_schedulers_online, int no_poll_th erts_no_dirty_io_schedulers = no_dirty_io_schedulers; erts_no_total_schedulers += no_dirty_io_schedulers; + /* normal schedulers */ + erts_no_aux_work_threads = n; + /* standard aux thread */ + erts_no_aux_work_threads += 1; + /* extra aux threads */ + erts_no_aux_work_threads += erts_no_dirty_alloc_instances; + /* Create and initialize scheduler sleep info */ - no_ssi = n + 1 /* aux thread */; aligned_sched_sleep_info = erts_alloc_permanent_cache_aligned( ERTS_ALC_T_SCHDLR_SLP_INFO, - no_ssi*sizeof(ErtsAlignedSchedulerSleepInfo)); - for (ix = 0; ix < no_ssi; ix++) { + erts_no_aux_work_threads*sizeof(ErtsAlignedSchedulerSleepInfo)); + for (ix = 0; ix < erts_no_aux_work_threads; ix++) { ErtsSchedulerSleepInfo *ssi = &aligned_sched_sleep_info[ix].ssi; #if 0 /* no need to initialize these... */ ssi->next = NULL; @@ -6067,7 +6151,7 @@ erts_init_scheduling(int no_schedulers, int no_schedulers_online, int no_poll_th /* Create and initialize scheduler specific data */ daww_sz = ERTS_ALC_CACHE_LINE_ALIGN_SIZE((sizeof(ErtsDelayedAuxWorkWakeupJob) - + sizeof(int))*(n+1)); + + sizeof(int))*erts_no_aux_work_threads); daww_ptr = erts_alloc_permanent_cache_aligned(ERTS_ALC_T_SCHDLR_DATA, daww_sz*n); @@ -6079,7 +6163,7 @@ erts_init_scheduling(int no_schedulers, int no_schedulers_online, int no_poll_th ErtsSchedulerData *esdp = ERTS_SCHEDULER_IX(ix); init_scheduler_data(esdp, ix+1, ERTS_SCHED_SLEEP_INFO_IX(ix), ERTS_RUNQ_IX(ix), &daww_ptr, daww_sz, - NULL, 0); + NULL, 0); } { @@ -6109,7 +6193,7 @@ erts_init_scheduling(int no_schedulers, int no_schedulers_online, int no_poll_th ErtsSchedulerData *esdp = ERTS_DIRTY_IO_SCHEDULER_IX(ix); init_scheduler_data(esdp, ix+1, ERTS_DIRTY_IO_SCHED_SLEEP_INFO_IX(ix), ERTS_DIRTY_IO_RUNQ, NULL, 0, - &adsp[adspix++].dsp, ts); + &adsp[adspix++].dsp, ts); } } @@ -6120,9 +6204,10 @@ erts_init_scheduling(int no_schedulers, int no_schedulers_online, int no_poll_th erts_atomic32_init_nob(&debug_wait_completed_count, 0); /* debug only */ debug_wait_completed_flags = 0; - aux_thread_aux_work_data = + aligned_aux_work_data = erts_alloc_permanent_cache_aligned(ERTS_ALC_T_SCHDLR_DATA, - sizeof(ErtsAuxWorkData)); + sizeof(ErtsAlignedAuxWorkData) + * (erts_no_dirty_alloc_instances + 1)); init_no_runqs(no_schedulers_online, no_schedulers_online); balance_info.last_active_runqs = no_schedulers; @@ -8539,7 +8624,6 @@ sched_thread_func(void *vesdp) esdp->ssi->psi = erts_create_pollset_thread(-1, NULL); #endif - erts_alloc_register_scheduler(vesdp); #ifdef ERTS_ENABLE_LOCK_CHECK { char buf[31]; @@ -8548,6 +8632,7 @@ sched_thread_func(void *vesdp) } #endif erts_tsd_set(sched_data_key, vesdp); + erts_alloc_register_scheduler(vesdp); #if HAVE_ERTS_MSEG erts_mseg_late_init(); #endif @@ -8564,7 +8649,6 @@ sched_thread_func(void *vesdp) ERTS_VERIFY_UNUSED_TEMP_ALLOC(NULL); #endif - erts_alcu_sched_spec_data_init(esdp); erts_ets_sched_spec_data_init(esdp); erts_utils_sched_spec_data_init(); @@ -8612,6 +8696,7 @@ sched_dirty_cpu_thread_func(void *vesdp) } #endif erts_tsd_set(sched_data_key, vesdp); + erts_alloc_register_scheduler(vesdp); esdp->aux_work_data.async_ready.queue = NULL; erts_proc_lock_prepare_proc_lock_waiter(); @@ -8659,6 +8744,7 @@ sched_dirty_io_thread_func(void *vesdp) } #endif erts_tsd_set(sched_data_key, vesdp); + erts_alloc_register_scheduler(vesdp); esdp->aux_work_data.async_ready.queue = NULL; erts_proc_lock_prepare_proc_lock_waiter(); @@ -8737,11 +8823,19 @@ erts_start_schedulers(void) } } - erts_snprintf(opts.name, sizeof(name), "aux"); + ix = 0; + while (ix < erts_no_aux_work_threads) { + int id = ix == 0 ? 1 : ix + 1 - (int) erts_no_schedulers; + erts_snprintf(opts.name, sizeof(name), "%d_aux", id); - res = ethr_thr_create(&tid, aux_thread, NULL, &opts); - if (res != 0) - erts_exit(ERTS_ABORT_EXIT, "Failed to create aux thread, error = %d\n", res); + res = ethr_thr_create(&tid, aux_thread, (void *) (Sint) ix, &opts); + if (res != 0) + erts_exit(ERTS_ABORT_EXIT, "Failed to create aux thread %d, error = %d\n", res); + if (ix == 0) + ix = (int) (1 + erts_no_schedulers); + else + ix++; + } block_poll_thread_data = (ErtsAlignedBlockPollThreadData *) erts_alloc_permanent_cache_aligned(ERTS_ALC_T_BLOCK_PTHR_DATA, @@ -8977,6 +9071,12 @@ erts_internal_is_process_executing_dirty_1(BIF_ALIST_1) BIF_RET(am_false); } +BIF_RETTYPE +erts_internal_no_aux_work_threads_0(BIF_ALIST_0) +{ + BIF_RET(make_small(erts_no_aux_work_threads)); +} + static ERTS_INLINE void run_queues_len_aux(ErtsRunQueue *rq, Uint *tot_len, Uint *qlen, int *ip, int incl_active_sched, int locked) { @@ -12499,7 +12599,8 @@ erl_create_process(Process* parent, /* Parent of process (default group leader). code = erts_dsig_prepare(&ctx, so->dist_entry, NULL, 0, ERTS_DSP_NO_LOCK, 1, 1, 0); - if (code == ERTS_DSIG_PREP_CONNECTED) { + if (code == ERTS_DSIG_PREP_CONNECTED + && ctx.connection_id == so->conn_id) { int dsflags = 0; if (so->flags & SPO_LINK) dsflags |= ERTS_DIST_SPAWN_FLAG_LINK; @@ -13103,7 +13204,7 @@ proc_exit_handle_pend_spawn_monitors(ErtsMonitor *mon, void *vctxt, Sint reds) item = copy_struct(mon->other.item, item_sz, &hp, factory.off_heap); erts_factory_close(&factory); code = erts_dsig_send_exit_tt(&ctx, - c_p->common.id, + c_p, item, reason, SEQ_TRACE_TOKEN(c_p)); @@ -13385,7 +13486,7 @@ erts_proc_exit_handle_dist_link(ErtsLink *lnk, void *vctxt, Sint reds) item = copy_struct(lnk->other.item, item_sz, &hp, factory.off_heap); erts_factory_close(&factory); code = erts_dsig_send_exit_tt(&ctx, - c_p->common.id, + c_p, item, reason, SEQ_TRACE_TOKEN(c_p)); @@ -13470,13 +13571,13 @@ erts_proc_exit_handle_link(ErtsLink *lnk, void *vctxt, Sint reds) if (!erts_link_dist_delete(dlnk)) elnk = NULL; - code = erts_dsig_prepare(&ctx, dep, c_p, 0, ERTS_DSP_NO_LOCK, 1, 1, 0); + code = erts_dsig_prepare(&ctx, dep, NULL, 0, ERTS_DSP_NO_LOCK, 1, 1, 0); switch (code) { case ERTS_DSIG_PREP_CONNECTED: case ERTS_DSIG_PREP_PENDING: if (dist->connection_id == ctx.connection_id) { code = erts_dsig_send_exit_tt(&ctx, - c_p->common.id, + c_p, lnk->other.item, reason, SEQ_TRACE_TOKEN(c_p)); diff --git a/erts/emulator/beam/erl_process.h b/erts/emulator/beam/erl_process.h index 6fa91a5049..56e5600e1b 100644 --- a/erts/emulator/beam/erl_process.h +++ b/erts/emulator/beam/erl_process.h @@ -106,6 +106,7 @@ extern Uint ERTS_WRITE_UNLIKELY(erts_no_total_schedulers); extern Uint ERTS_WRITE_UNLIKELY(erts_no_dirty_cpu_schedulers); extern Uint ERTS_WRITE_UNLIKELY(erts_no_dirty_io_schedulers); extern Uint ERTS_WRITE_UNLIKELY(erts_no_run_queues); +extern int ERTS_WRITE_UNLIKELY(erts_no_aux_work_threads); extern int erts_sched_thread_suggested_stack_size; extern int erts_dcpu_sched_thread_suggested_stack_size; extern int erts_dio_sched_thread_suggested_stack_size; @@ -558,8 +559,9 @@ typedef struct { erts_aint32_t aux_work; } ErtsDelayedAuxWorkWakeupJob; -typedef struct { - int sched_id; +typedef struct ErtsAuxWorkData_ { + int aux_work_tid; + ErtsThrAllocData alloc_data; ErtsSchedulerData *esdp; ErtsSchedulerSleepInfo *ssi; ErtsThrPrgrVal current_thr_prgr; @@ -607,7 +609,8 @@ typedef struct { #define ERTS_SCHED_AUX_YIELD_DATA(ESDP, NAME) \ (&(ESDP)->aux_work_data.yield.NAME) -void erts_notify_new_aux_yield_work(ErtsSchedulerData *esdp); +void erts_more_yield_aux_work(ErtsAuxWorkData *); +ErtsAuxWorkData *erts_get_aux_work_data(void); typedef enum { ERTS_DIRTY_CPU_SCHEDULER, @@ -707,8 +710,6 @@ struct ErtsSchedulerData_ { Uint64 unique; Uint64 ref; - ErtsSchedAllocData alloc_data; - struct { Uint64 out; Uint64 in; @@ -1434,6 +1435,7 @@ typedef struct { Eterm group_leader; Eterm mfa; DistEntry *dist_entry; + Uint32 conn_id; ErtsMonLnkDist *mld; /* copied from dist_entry->mld */ ErtsDistExternal *edep; ErlHeapFragment *ede_hfrag; @@ -1938,7 +1940,8 @@ void erts_schedule_misc_aux_work(int sched_id, void (*func)(void *), void *arg); void erts_schedule_multi_misc_aux_work(int ignore_self, - int max_sched, + int min_tid, + int max_tid, void (*func)(void *), void *arg); erts_aint32_t erts_set_aux_work_timeout(int, erts_aint32_t, int); diff --git a/erts/emulator/beam/erl_term.h b/erts/emulator/beam/erl_term.h index 8b2b82e527..d7a22c7acc 100644 --- a/erts/emulator/beam/erl_term.h +++ b/erts/emulator/beam/erl_term.h @@ -955,9 +955,10 @@ typedef union { ((*((Eterm *)(x)) == ERTS_REF_THING_HEADER) \ & (((ErtsRefThing *) (x))->o.marker == ERTS_ORDINARY_REF_MARKER)) -#define is_magic_ref_thing(x) \ - ((*((Eterm *)(x)) == ERTS_REF_THING_HEADER) \ - & (((ErtsRefThing *) (x))->o.marker != ERTS_ORDINARY_REF_MARKER)) +/* the _with_hdr variant usable when header word may be broken (copy_shared) */ +#define is_magic_ref_thing_with_hdr(PTR,HDR) \ + (((HDR) == ERTS_REF_THING_HEADER) \ + & (((ErtsRefThing *) (PTR))->o.marker != ERTS_ORDINARY_REF_MARKER)) #else /* Ordinary and magic references of different sizes... */ @@ -970,11 +971,14 @@ typedef union { #define is_pid_ref_thing(x) \ (*((Eterm *)(x)) == ERTS_PID_REF_THING_HEADER) -#define is_magic_ref_thing(x) \ - (*((Eterm *)(x)) == ERTS_MAGIC_REF_THING_HEADER) +#define is_magic_ref_thing_with_hdr(PTR,HDR) \ + ((HDR) == ERTS_MAGIC_REF_THING_HEADER) #endif +#define is_magic_ref_thing(PTR) \ + is_magic_ref_thing_with_hdr(PTR, *(Eterm *)(PTR)) + #define is_internal_magic_ref(x) \ (_unchecked_is_boxed((x)) && is_magic_ref_thing(boxed_val((x)))) diff --git a/erts/emulator/beam/external.c b/erts/emulator/beam/external.c index f4c7f6dec4..72c7ead6ae 100644 --- a/erts/emulator/beam/external.c +++ b/erts/emulator/beam/external.c @@ -3039,7 +3039,7 @@ dec_pid(ErtsDistExternal *edep, ErtsHeapFactory* factory, const byte* ep, #define ENC_START_SORTING_MAP ((Eterm) 7) #define ENC_CONTINUE_SORTING_MAP ((Eterm) 8) #define ENC_PUSH_SORTED_MAP ((Eterm) 9) -#define ENC_LAST_ARRAY_ELEMENT ((Eterm) 10) +#define ENC_LAST_ARRAY_ELEMENT ((Eterm) 10) /* must be the largest one */ static Eterm* alloc_map_array(Uint size) { @@ -3466,6 +3466,7 @@ enc_term_int(TTBEncodeContext* ctx, ErtsAtomCacheMap *acmp, Eterm obj, byte* ep, ep += 4; } if (i > 0) { + ASSERT(ENC_LAST_ARRAY_ELEMENT+i-1 >= ENC_LAST_ARRAY_ELEMENT); WSTACK_PUSH2(s, ENC_LAST_ARRAY_ELEMENT+i-1, (UWord)ptr); } break; diff --git a/erts/emulator/beam/generators.tab b/erts/emulator/beam/generators.tab index 520c688113..9c9d357475 100644 --- a/erts/emulator/beam/generators.tab +++ b/erts/emulator/beam/generators.tab @@ -93,6 +93,19 @@ gen.get_float2(Fail, Ms, Live, Size, Unit, Flags, Dst) { return op; } +gen.get_utf16(Fail, Ms, Flags, Dst) { + BeamOp* op; + $NewBeamOp(S, op); + + $NativeEndian(Flags); + $BeamOpNameArity(op, i_bs_get_utf16, 4); + op->a[0] = Ms; + op->a[1] = Fail; + op->a[2] = Flags; + op->a[3] = Dst; + return op; +} + gen.put_binary(Fail, Size, Unit, Flags, Src) { BeamOp* op; $NewBeamOp(S, op); @@ -234,6 +247,18 @@ gen.put_float(Fail, Size, Unit, Flags, Src) { return op; } +gen.put_utf16(Fail, Flags, Src) { + BeamOp* op; + $NewBeamOp(S, op); + + $NativeEndian(Flags); + $BeamOpNameArity(op, i_bs_put_utf16, 3); + op->a[0] = Fail; + op->a[1] = Flags; + op->a[2] = Src; + return op; +} + // Generate the fastest instruction for bs_skip_bits. gen.skip_bits2(Fail, Ms, Size, Unit, Flags) { BeamOp* op; diff --git a/erts/emulator/beam/global.h b/erts/emulator/beam/global.h index 904d394fca..9176309e42 100644 --- a/erts/emulator/beam/global.h +++ b/erts/emulator/beam/global.h @@ -443,28 +443,29 @@ do { \ #define ESTACK_IS_STATIC(s) ((s).start == ESTK_DEF_STACK(s)) +#define ESTACK_RESERVE(s, push_cnt) \ +do { \ + if ((s).end - (s).sp < (Sint)(push_cnt)) { \ + erl_grow_estack(&(s), (push_cnt)); \ + } \ +} while(0) + #define ESTACK_PUSH(s, x) \ do { \ - if ((s).sp == (s).end) { \ - erl_grow_estack(&(s), 1); \ - } \ + ESTACK_RESERVE(s, 1); \ *(s).sp++ = (x); \ } while(0) #define ESTACK_PUSH2(s, x, y) \ do { \ - if ((s).sp > (s).end - 2) { \ - erl_grow_estack(&(s), 2); \ - } \ + ESTACK_RESERVE(s, 2); \ *(s).sp++ = (x); \ *(s).sp++ = (y); \ } while(0) #define ESTACK_PUSH3(s, x, y, z) \ do { \ - if ((s).sp > (s).end - 3) { \ - erl_grow_estack(&s, 3); \ - } \ + ESTACK_RESERVE(s, 3); \ *(s).sp++ = (x); \ *(s).sp++ = (y); \ *(s).sp++ = (z); \ @@ -472,22 +473,13 @@ do { \ #define ESTACK_PUSH4(s, E1, E2, E3, E4) \ do { \ - if ((s).sp > (s).end - 4) { \ - erl_grow_estack(&s, 4); \ - } \ + ESTACK_RESERVE(s, 4); \ *(s).sp++ = (E1); \ *(s).sp++ = (E2); \ *(s).sp++ = (E3); \ *(s).sp++ = (E4); \ } while(0) -#define ESTACK_RESERVE(s, push_cnt) \ -do { \ - if ((s).sp > (s).end - (push_cnt)) { \ - erl_grow_estack(&(s), (push_cnt)); \ - } \ -} while(0) - /* Must be preceded by ESTACK_RESERVE */ #define ESTACK_FAST_PUSH(s, x) \ do { \ @@ -642,28 +634,29 @@ do { \ #define WSTACK_IS_STATIC(s) (s.wstart == WSTK_DEF_STACK(s)) -#define WSTACK_PUSH(s, x) \ -do { \ - if (s.wsp == s.wend) { \ - erl_grow_wstack(&s, 1); \ - } \ - *s.wsp++ = (x); \ +#define WSTACK_RESERVE(s, push_cnt) \ +do { \ + if (s.wend - s.wsp < (Sint)(push_cnt)) { \ + erl_grow_wstack(&s, (push_cnt)); \ + } \ +} while(0) + +#define WSTACK_PUSH(s, x) \ +do { \ + WSTACK_RESERVE(s, 1); \ + *s.wsp++ = (x); \ } while(0) #define WSTACK_PUSH2(s, x, y) \ do { \ - if (s.wsp > s.wend - 2) { \ - erl_grow_wstack(&s, 2); \ - } \ + WSTACK_RESERVE(s, 2); \ *s.wsp++ = (x); \ *s.wsp++ = (y); \ } while(0) #define WSTACK_PUSH3(s, x, y, z) \ do { \ - if (s.wsp > s.wend - 3) { \ - erl_grow_wstack(&s, 3); \ - } \ + WSTACK_RESERVE(s, 3); \ *s.wsp++ = (x); \ *s.wsp++ = (y); \ *s.wsp++ = (z); \ @@ -671,9 +664,7 @@ do { \ #define WSTACK_PUSH4(s, A1, A2, A3, A4) \ do { \ - if (s.wsp > s.wend - 4) { \ - erl_grow_wstack(&s, 4); \ - } \ + WSTACK_RESERVE(s, 4); \ *s.wsp++ = (A1); \ *s.wsp++ = (A2); \ *s.wsp++ = (A3); \ @@ -682,9 +673,7 @@ do { \ #define WSTACK_PUSH5(s, A1, A2, A3, A4, A5) \ do { \ - if (s.wsp > s.wend - 5) { \ - erl_grow_wstack(&s, 5); \ - } \ + WSTACK_RESERVE(s, 5); \ *s.wsp++ = (A1); \ *s.wsp++ = (A2); \ *s.wsp++ = (A3); \ @@ -694,9 +683,7 @@ do { \ #define WSTACK_PUSH6(s, A1, A2, A3, A4, A5, A6) \ do { \ - if (s.wsp > s.wend - 6) { \ - erl_grow_wstack(&s, 6); \ - } \ + WSTACK_RESERVE(s, 6); \ *s.wsp++ = (A1); \ *s.wsp++ = (A2); \ *s.wsp++ = (A3); \ @@ -705,13 +692,6 @@ do { \ *s.wsp++ = (A6); \ } while(0) -#define WSTACK_RESERVE(s, push_cnt) \ -do { \ - if (s.wsp > s.wend - (push_cnt)) { \ - erl_grow_wstack(&s, (push_cnt)); \ - } \ -} while(0) - /* Must be preceded by WSTACK_RESERVE */ #define WSTACK_FAST_PUSH(s, x) \ do { \ diff --git a/erts/emulator/beam/io.c b/erts/emulator/beam/io.c index 477f5bd749..f13a7d437a 100644 --- a/erts/emulator/beam/io.c +++ b/erts/emulator/beam/io.c @@ -4947,6 +4947,7 @@ erts_request_io_bytes(Process *c_p) if (erts_no_schedulers > 1) erts_schedule_multi_misc_aux_work(1, + 1, erts_no_schedulers, reply_io_bytes, (void *) req); diff --git a/erts/emulator/beam/jit/beam_jit_common.c b/erts/emulator/beam/jit/beam_jit_common.c index a69a48beff..d2d4ad43d6 100644 --- a/erts/emulator/beam/jit/beam_jit_common.c +++ b/erts/emulator/beam/jit/beam_jit_common.c @@ -664,8 +664,7 @@ Sint beam_jit_remove_message(Process *c_p, tok_serial); } #endif - erts_msgq_unlink_msg(c_p, msgp); - erts_msgq_set_save_first(c_p); + erts_msgq_unlink_msg_set_save_first(c_p, msgp); CANCEL_TIMER(c_p); erts_save_message_in_proc(c_p, msgp); diff --git a/erts/emulator/beam/jit/x86/generators.tab b/erts/emulator/beam/jit/x86/generators.tab index ab755fde4c..e09c9b5c7b 100644 --- a/erts/emulator/beam/jit/x86/generators.tab +++ b/erts/emulator/beam/jit/x86/generators.tab @@ -451,6 +451,18 @@ gen.get_binary2(Fail, Ms, Live, Size, Unit, Flags, Dst) { return op; } +gen.skip_utf16(Fail, Ms, Flags) { + BeamOp* op; + $NewBeamOp(S, op); + + $NativeEndian(Flags); + $BeamOpNameArity(op, i_bs_skip_utf16, 3); + op->a[0] = Ms; + op->a[1] = Fail; + op->a[2] = Flags; + return op; +} + gen.combine_conses(Len, Dst, Hd) { BeamOp* cons; BeamOp* tail; diff --git a/erts/emulator/beam/jit/x86/instr_arith.cpp b/erts/emulator/beam/jit/x86/instr_arith.cpp index 5615230551..56e2fa59da 100644 --- a/erts/emulator/beam/jit/x86/instr_arith.cpp +++ b/erts/emulator/beam/jit/x86/instr_arith.cpp @@ -417,14 +417,13 @@ void BeamGlobalAssembler::emit_int_div_rem_guard_shared() { emit_leave_runtime(); + /* erts_int_div returns 0 on failure and 1 on success. */ + a.test(RETd, RETd); + /* Place the result in RAX:RDX, mirroring the `idiv` instruction. */ a.mov(x86::rax, TMP_MEM1q); a.mov(x86::rdx, TMP_MEM2q); - /* erts_int_div returns a tagged value, so we know it's non-zero and can - * clear ZF by and it with itself. */ - a.test(RET, RET); - /* Fall through */ } @@ -498,7 +497,8 @@ void BeamGlobalAssembler::emit_int_div_rem_body_shared() { emit_leave_runtime(); - a.test(RET, RET); + /* erts_int_div returns 0 on failure and 1 on success. */ + a.test(RETd, RETd); /* Place the result in RAX:RDX, mirroring the `idiv` instruction. */ a.mov(x86::rax, TMP_MEM4q); diff --git a/erts/emulator/beam/jit/x86/instr_bs.cpp b/erts/emulator/beam/jit/x86/instr_bs.cpp index 7df0b84cae..3dc6894efa 100644 --- a/erts/emulator/beam/jit/x86/instr_bs.cpp +++ b/erts/emulator/beam/jit/x86/instr_bs.cpp @@ -1224,9 +1224,9 @@ void BeamModuleAssembler::emit_i_bs_utf16_size(const ArgVal &Src, mov_arg(Dst, RET); } -void BeamModuleAssembler::emit_bs_put_utf16(const ArgVal &Fail, - const ArgVal &Flags, - const ArgVal &Src) { +void BeamModuleAssembler::emit_i_bs_put_utf16(const ArgVal &Fail, + const ArgVal &Flags, + const ArgVal &Src) { Label next; if (Fail.getValue() == 0) { @@ -1407,8 +1407,8 @@ void BeamModuleAssembler::emit_bs_add(const ArgVal &Fail, a.and_(RETd, ARG1d); } } - a.and_(RETb, imm(_TAG_PRIMARY_MASK)); - a.cmp(RETb, imm(TAG_PRIMARY_IMMED1)); + a.and_(RETb, imm(_TAG_IMMED1_MASK)); + a.cmp(RETb, imm(_TAG_IMMED1_SMALL)); a.jne(fail); /* Verify that ARG2 >= 0 and multiply ARG2 by the unit. The diff --git a/erts/emulator/beam/jit/x86/ops.tab b/erts/emulator/beam/jit/x86/ops.tab index d6af3818a1..8ccfb6cfbc 100644 --- a/erts/emulator/beam/jit/x86/ops.tab +++ b/erts/emulator/beam/jit/x86/ops.tab @@ -886,8 +886,8 @@ i_bs_get_utf8 S f? d bs_skip_utf8 Fail=f Ms=xy u u => i_bs_skip_utf8 Ms Fail i_bs_skip_utf8 S f? -bs_get_utf16 Fail=f Ms=xy u Flags=u Dst=d => i_bs_get_utf16 Ms Fail Flags Dst -bs_skip_utf16 Fail=f Ms=xy u Flags=u => i_bs_skip_utf16 Ms Fail Flags +bs_get_utf16 Fail=f Ms=xy u Flags=u Dst=d => get_utf16(Fail, Ms, Flags, Dst) +bs_skip_utf16 Fail=f Ms=xy u Flags=u => skip_utf16(Fail, Ms, Flags) i_bs_get_utf16 S f? t d i_bs_skip_utf16 S f? t @@ -983,6 +983,7 @@ bs_utf8_size j Src Dst=d => i_bs_utf8_size Src Dst bs_utf16_size j Src Dst=d => i_bs_utf16_size Src Dst bs_put_utf8 Fail u Src => i_bs_put_utf8 Fail Src +bs_put_utf16 Fail Flags Src => put_utf16(Fail, Flags, Src) bs_put_utf32 Fail=j Flags=u Src=s => \ i_bs_validate_unicode Fail Src | bs_put_integer Fail i=32 u=1 Flags Src @@ -991,7 +992,7 @@ i_bs_utf8_size s x i_bs_utf16_size s x i_bs_put_utf8 j? s -bs_put_utf16 j? t s +i_bs_put_utf16 j? t s i_bs_validate_unicode j? s diff --git a/erts/emulator/drivers/common/inet_drv.c b/erts/emulator/drivers/common/inet_drv.c index b69906b3e1..00421d0c73 100644 --- a/erts/emulator/drivers/common/inet_drv.c +++ b/erts/emulator/drivers/common/inet_drv.c @@ -1039,26 +1039,26 @@ typedef union { (1 + 2 + 4) : localaddrlen(data)) #endif +typedef int (MultiTimerFunction)(ErlDrvData drv_data, ErlDrvTermData caller); + typedef struct _multi_timer_data { ErlDrvTime when; ErlDrvTermData caller; - void (*timeout_function)(ErlDrvData drv_data, ErlDrvTermData caller); + MultiTimerFunction *timeout_function; struct _multi_timer_data *next; struct _multi_timer_data *prev; } MultiTimerData; static MultiTimerData *add_multi_timer(tcp_descriptor *desc, ErlDrvPort port, ErlDrvTermData caller, unsigned timeout, - void (*timeout_fun)(ErlDrvData drv_data, - ErlDrvTermData caller)); + MultiTimerFunction *timeout_fun); static void fire_multi_timers(tcp_descriptor *desc, ErlDrvPort port, ErlDrvData data); static void remove_multi_timer(tcp_descriptor *desc, ErlDrvPort port, MultiTimerData *p); static void cancel_multi_timer(tcp_descriptor *desc, ErlDrvPort port, - void (*timeout_fun)(ErlDrvData drv_data, - ErlDrvTermData caller)); + MultiTimerFunction *timeout_fun); -static void tcp_inet_multi_timeout(ErlDrvData e, ErlDrvTermData caller); +static int tcp_inet_multi_timeout(ErlDrvData e, ErlDrvTermData caller); static void clean_multi_timers(tcp_descriptor *desc, ErlDrvPort port); typedef struct { @@ -10131,13 +10131,14 @@ static void tcp_desc_close(tcp_descriptor* desc) erl_inet_close(INETP(desc)); } -static void tcp_inet_recv_timeout(ErlDrvData e, ErlDrvTermData dummy) +static int tcp_inet_recv_timeout(ErlDrvData e, ErlDrvTermData dummy) { tcp_descriptor* desc = (tcp_descriptor*)e; ASSERT(!desc->inet.active); sock_select(INETP(desc),(FD_READ|FD_CLOSE),0); desc->i_remain = 0; async_error_am(INETP(desc), am_timeout); + return 1; } /* TCP requests from Erlang */ @@ -10554,7 +10555,7 @@ static ErlDrvSSizeT tcp_inet_ctl(ErlDrvData e, unsigned int cmd, } -static void tcp_inet_send_timeout(ErlDrvData e, ErlDrvTermData dummy) +static int tcp_inet_send_timeout(ErlDrvData e, ErlDrvTermData dummy) { tcp_descriptor* desc = (tcp_descriptor*)e; ASSERT(IS_BUSY(INETP(desc))); @@ -10567,6 +10568,7 @@ static void tcp_inet_send_timeout(ErlDrvData e, ErlDrvTermData dummy) if (desc->send_timeout_close) { tcp_desc_close(desc); } + return 1; /* Q: Why not keep port busy as send queue may still be full (ERL-1390)? * * A: If kept busy, a following send call would hang without a timeout @@ -10616,14 +10618,14 @@ static void tcp_inet_timeout(ErlDrvData e) DEBUGF(("tcp_inet_timeout(%p) }\r\n", desc->inet.port)); } -static void tcp_inet_multi_timeout(ErlDrvData e, ErlDrvTermData caller) +static int tcp_inet_multi_timeout(ErlDrvData e, ErlDrvTermData caller) { tcp_descriptor* desc = (tcp_descriptor*)e; int id,req; ErlDrvMonitor monitor; if (remove_multi_op(desc, &id, &req, caller, NULL, &monitor) != 0) { - return; + return 1; } driver_demonitor_process(desc->inet.port, &monitor); if (desc->multi_first == NULL) { @@ -10631,6 +10633,7 @@ static void tcp_inet_multi_timeout(ErlDrvData e, ErlDrvTermData caller) desc->inet.state = INET_STATE_LISTENING; /* restore state */ } send_async_error(desc->inet.dport, id, caller, am_timeout); + return 1; } @@ -11605,10 +11608,10 @@ static int tcp_shutdown_error(tcp_descriptor* desc, int err) return tcp_send_or_shutdown_error(desc, err); } -static void tcp_inet_delay_send(ErlDrvData data, ErlDrvTermData dummy) +static int tcp_inet_delay_send(ErlDrvData data, ErlDrvTermData dummy) { tcp_descriptor *desc = (tcp_descriptor*)data; - (void)tcp_inet_output(desc, (HANDLE) INETP(desc)->s); + return tcp_inet_output(desc, (HANDLE) INETP(desc)->s); } /* @@ -12632,15 +12635,19 @@ static ErlDrvSSizeT packet_inet_ctl(ErlDrvData e, unsigned int cmd, char* buf, * change the source address when connecting * a datagram socket to a new destination */ - sock_connect(desc->s, &disassoc_sa, disassoc_sa_size); + if (IS_CONNECTED(desc)) { + /* Dissolve association */ + (void) sock_connect(desc->s, &disassoc_sa, disassoc_sa_size); + } #endif /* #ifdef __linux__ */ code = sock_connect(desc->s, (struct sockaddr*) &desc->remote, len); if (IS_SOCKET_ERROR(code)) { - sock_connect(desc->s, &disassoc_sa, disassoc_sa_size); + code = sock_errno(), + (void) sock_connect(desc->s, &disassoc_sa, disassoc_sa_size); desc->state &= ~INET_F_ACTIVE; - return ctl_error(sock_errno(), rbuf, rsize); + return ctl_error(code, rbuf, rsize); } else /* ok we are connected */ { enq_async(desc, tbuf, INET_REQ_CONNECT); @@ -13320,7 +13327,10 @@ static void fire_multi_timers(tcp_descriptor *desc, ErlDrvPort port, if (desc->mtd != NULL) desc->mtd->prev = NULL; - (*(save.timeout_function))(data,save.caller); + if ((*(save.timeout_function))(data,save.caller) < 0) + /* -1 means tcp_descriptor has been deallocated, + so we have to return as soon as possible */ + return; if (desc->mtd == NULL) return; @@ -13372,8 +13382,7 @@ static void remove_multi_timer(tcp_descriptor *desc, ErlDrvPort port, MultiTimer /* Cancel a timer based on the timeout_fun */ static void cancel_multi_timer(tcp_descriptor *desc, ErlDrvPort port, - void (*timeout_fun)(ErlDrvData drv_data, - ErlDrvTermData caller)) + MultiTimerFunction *timeout_fun) { MultiTimerData *timer = desc->mtd; while(timer && timer->timeout_function != timeout_fun) { @@ -13386,8 +13395,7 @@ static void cancel_multi_timer(tcp_descriptor *desc, ErlDrvPort port, static MultiTimerData *add_multi_timer(tcp_descriptor *desc, ErlDrvPort port, ErlDrvTermData caller, unsigned timeout, - void (*timeout_fun)(ErlDrvData drv_data, - ErlDrvTermData caller)) + MultiTimerFunction *timeout_fun) { MultiTimerData *mtd, *p, *s; diff --git a/erts/emulator/nifs/common/prim_net_nif.c b/erts/emulator/nifs/common/prim_net_nif.c index cff7031250..f9cd62f4e2 100644 --- a/erts/emulator/nifs/common/prim_net_nif.c +++ b/erts/emulator/nifs/common/prim_net_nif.c @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson AB 2018-2020. All Rights Reserved. + * Copyright Ericsson AB 2018-2021. All Rights Reserved. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. @@ -438,7 +438,6 @@ static const struct in6_addr in6addr_loopback = LOCAL_ATOM_DECL(idn); \ LOCAL_ATOM_DECL(master); \ LOCAL_ATOM_DECL(multicast); \ - LOCAL_ATOM_DECL(name); \ LOCAL_ATOM_DECL(namereqd); \ LOCAL_ATOM_DECL(name_info); \ LOCAL_ATOM_DECL(netmask); \ @@ -1219,9 +1218,9 @@ ERL_NIF_TERM enet_getifaddrs_process(ErlNifEnv* env, struct ifaddrs* ifap) NDBG( ("NET", "enet_getifaddrs_process -> len: %d\r\n", len) ); if (len > 0) { - ERL_NIF_TERM* array = MALLOC(len * sizeof(ERL_NIF_TERM)); - unsigned int i = 0; - struct ifaddrs* p = ifap; + ERL_NIF_TERM* array = MALLOC(len * sizeof(ERL_NIF_TERM)); + unsigned int i = 0; + struct ifaddrs* p = ifap; while (i < len) { ERL_NIF_TERM entry; @@ -1328,7 +1327,7 @@ void encode_ifaddrs(ErlNifEnv* env, static ERL_NIF_TERM encode_ifaddrs_name(ErlNifEnv* env, char* name) { - return ((name == NULL) ? esock_atom_undefined : MKS(env, name)); + return ((name == NULL) ? esock_atom_undefined : MKS(env, name)); } @@ -1460,7 +1459,7 @@ void make_ifaddrs(ErlNifEnv* env, /* *** Name *** */ NDBG( ("NET", "make_ifaddrs -> name: %T\r\n", ename) ); - keys[idx] = atom_name; + keys[idx] = esock_atom_name; vals[idx] = ename; idx++; diff --git a/erts/emulator/nifs/common/prim_socket_nif.c b/erts/emulator/nifs/common/prim_socket_nif.c index 23015ccdcc..249917c1aa 100644 --- a/erts/emulator/nifs/common/prim_socket_nif.c +++ b/erts/emulator/nifs/common/prim_socket_nif.c @@ -620,6 +620,282 @@ static const struct msg_flag { }; +static const struct ioctl_flag { + int flag; + ERL_NIF_TERM *name; +} ioctl_flags[] = { + + { +#ifdef IFF_UP + IFF_UP, +#else + 0, +#endif + &esock_atom_up}, + + { +#ifdef IFF_BROADCAST + IFF_BROADCAST, +#else + 0, +#endif + &esock_atom_broadcast}, + + { +#ifdef IFF_DEBUG + IFF_DEBUG, +#else + 0, +#endif + &esock_atom_debug}, + + { +#ifdef IFF_LOOPBACK + IFF_LOOPBACK, +#else + 0, +#endif + &esock_atom_loopback}, + + { +#ifdef IFF_POINTOPOINT + IFF_POINTOPOINT, +#else + 0, +#endif + &esock_atom_pointopoint}, + + { +#ifdef IFF_NOTRAILERS + IFF_NOTRAILERS, +#else + 0, +#endif + &esock_atom_notrailers}, + + /* FreeBSD: Has the same value as (Linux) notrailers */ + { +#ifdef IFF_KNOWSEPOCH + IFF_KNOWSEPOCH, +#else + 0, +#endif + &esock_atom_knowsepoch}, + + { +#ifdef IFF_RUNNING + IFF_RUNNING, +#else + 0, +#endif + &esock_atom_running}, + + { +#ifdef IFF_NOARP + IFF_NOARP, +#else + 0, +#endif + &esock_atom_noarp}, + + { +#ifdef IFF_PROMISC + IFF_PROMISC, +#else + 0, +#endif + &esock_atom_promisc}, + + { +#ifdef IFF_ALLMULTI + IFF_ALLMULTI, +#else + 0, +#endif + &esock_atom_allmulti}, + + { +#ifdef IFF_MASTER + IFF_MASTER, +#else + 0, +#endif + &esock_atom_master}, + + /* FreeBSD: Has the same value as (Linux) master */ + { +#ifdef IFF_OACTIVE + IFF_OACTIVE, +#else + 0, +#endif + &esock_atom_oactive}, + + { +#ifdef IFF_SLAVE + IFF_SLAVE, +#else + 0, +#endif + &esock_atom_slave}, + + /* FreeBSD: Has the same value as (Linux) slave */ + { +#ifdef IFF_SIMPLEX + IFF_SIMPLEX, +#else + 0, +#endif + &esock_atom_simplex}, + + { +#ifdef IFF_MULTICAST + IFF_MULTICAST, +#else + 0, +#endif + &esock_atom_multicast}, + + // FreeBSD, ... + { +#ifdef IFF_LINK0 + IFF_LINK0, +#else + 0, +#endif + &esock_atom_link0}, + + // FreeBSD, ... + { +#ifdef IFF_LINK1 + IFF_LINK1, +#else + 0, +#endif + &esock_atom_link1}, + + // FreeBSD, ... + { +#ifdef IFF_LINK2 + IFF_LINK2, +#else + 0, +#endif + &esock_atom_link2}, + + { +#ifdef IFF_PORTSEL + IFF_PORTSEL, +#else + 0, +#endif + &esock_atom_portsel}, + + { +#ifdef IFF_AUTOMEDIA + IFF_AUTOMEDIA, +#else + 0, +#endif + &esock_atom_automedia}, + + { +#ifdef IFF_DYNAMIC + IFF_DYNAMIC, +#else + 0, +#endif + &esock_atom_dynamic}, + + // FreeBSD, ... + { +#ifdef IFF_CANTCONFIG + IFF_CANTCONFIG, +#else + 0, +#endif + &esock_atom_cantconfig}, + + { +#ifdef IFF_LOWER_UP + IFF_LOWER_UP, +#else + 0, +#endif + &esock_atom_lower_up}, + + // FreeBSD, ... + { +#ifdef IFF_PPROMISC + IFF_PPROMISC, +#else + 0, +#endif + &esock_atom_ppromisc}, + + { +#ifdef IFF_DORMANT + IFF_DORMANT, +#else + 0, +#endif + &esock_atom_dormant}, + + // FreeBSD, ... + { +#ifdef IFF_MONITOR + IFF_MONITOR, +#else + 0, +#endif + &esock_atom_monitor}, + + { +#ifdef IFF_ECHO + IFF_ECHO, +#else + 0, +#endif + &esock_atom_echo}, + + // FreeBSD, ... + { +#ifdef IFF_STATICARP + IFF_STATICARP, +#else + 0, +#endif + &esock_atom_staticarp}, + + // FreeBSD, ... + { +#ifdef IFF_DYING + IFF_DYING, +#else + 0, +#endif + &esock_atom_dying}, + + // FreeBSD, ... + { +#ifdef IFF_RENAMING + IFF_RENAMING, +#else + 0, +#endif + &esock_atom_renaming}, + + // FreeBSD, ... + { +#ifdef IFF_NOGROUP + IFF_NOGROUP, +#else + 0, +#endif + &esock_atom_nogroup} +}; + + /* level 'otp' options */ #define ESOCK_OPT_OTP_DEBUG 1001 @@ -1093,6 +1369,7 @@ extern char* erl_errno_id(int error); /* THIS IS JUST TEMPORARY??? */ * nif_getopt * nif_sockname * nif_peername + * nif_ioctl * nif_finalize_close * nif_cancel */ @@ -1118,6 +1395,7 @@ extern char* erl_errno_id(int error); /* THIS IS JUST TEMPORARY??? */ ESOCK_NIF_FUNC_DEF(getopt); \ ESOCK_NIF_FUNC_DEF(sockname); \ ESOCK_NIF_FUNC_DEF(peername); \ + ESOCK_NIF_FUNC_DEF(ioctl); \ ESOCK_NIF_FUNC_DEF(finalize_close); \ ESOCK_NIF_FUNC_DEF(cancel); @@ -1184,6 +1462,8 @@ static ERL_NIF_TERM esock_supports_1(ErlNifEnv* env, ERL_NIF_TERM key); static ERL_NIF_TERM esock_supports_msg_flags(ErlNifEnv* env); static ERL_NIF_TERM esock_supports_protocols(ErlNifEnv* env); +static ERL_NIF_TERM esock_supports_ioctl_requests(ErlNifEnv* env); +static ERL_NIF_TERM esock_supports_ioctl_flags(ErlNifEnv* env); static ERL_NIF_TERM esock_supports_options(ErlNifEnv* env); static ERL_NIF_TERM esock_open2(ErlNifEnv* env, @@ -1721,6 +2001,243 @@ static ERL_NIF_TERM esock_sockname(ErlNifEnv* env, ESockDescriptor* descP); static ERL_NIF_TERM esock_peername(ErlNifEnv* env, ESockDescriptor* descP); + +static ERL_NIF_TERM esock_ioctl1(ErlNifEnv* env, + ESockDescriptor* descP, + unsigned long req); +static ERL_NIF_TERM esock_ioctl2(ErlNifEnv* env, + ESockDescriptor* descP, + unsigned long req, + ERL_NIF_TERM arg); +static ERL_NIF_TERM esock_ioctl3(ErlNifEnv* env, + ESockDescriptor* descP, + unsigned long req, + ERL_NIF_TERM ename, + ERL_NIF_TERM eval); +static ERL_NIF_TERM esock_ioctl_gifconf(ErlNifEnv* env, + ESockDescriptor* descP); +#if defined(SIOCGIFNAME) +static ERL_NIF_TERM esock_ioctl_gifname(ErlNifEnv* env, + ESockDescriptor* descP, + ERL_NIF_TERM eidx); +#endif + +/* esock_ioctl_gifindex */ +#if defined(SIOCGIFINDEX) +#define IOCTL_GIFINDEX_FUNC_DEF IOCTL_GET_FUNC_DEF(gifindex) +#else +#define IOCTL_GIFINDEX_FUNC_DEF +#endif + +/* esock_ioctl_gifflags */ +#if defined(SIOCGIFFLAGS) +#define IOCTL_GIFFLAGS_FUNC_DEF IOCTL_GET_FUNC_DEF(gifflags) +#else +#define IOCTL_GIFFLAGS_FUNC_DEF +#endif + +/* esock_ioctl_gifaddr */ +#if defined(SIOCGIFADDR) +#define IOCTL_GIFADDR_FUNC_DEF IOCTL_GET_FUNC_DEF(gifaddr) +#else +#define IOCTL_GIFADDR_FUNC_DEF +#endif + +/* esock_ioctl_gifdstaddr */ +#if defined(SIOCGIFDSTADDR) +#define IOCTL_GIFDSTADDR_FUNC_DEF IOCTL_GET_FUNC_DEF(gifdstaddr) +#else +#define IOCTL_GIFDSTADDR_FUNC_DEF +#endif + +/* esock_ioctl_gifbrdaddr */ +#if defined(SIOCGIFBRDADDR) +#define IOCTL_GIFBRDADDR_FUNC_DEF IOCTL_GET_FUNC_DEF(gifbrdaddr) +#else +#define IOCTL_GIFBRDADDR_FUNC_DEF +#endif + +/* esock_ioctl_gifnetmask */ +#if defined(SIOCGIFNETMASK) +#define IOCTL_GIFNETMASK_FUNC_DEF IOCTL_GET_FUNC_DEF(gifnetmask) +#else +#define IOCTL_GIFNETMASK_FUNC_DEF +#endif + +/* esock_ioctl_gifmtu */ +#if defined(SIOCGIFMTU) +#define IOCTL_GIFMTU_FUNC_DEF IOCTL_GET_FUNC_DEF(gifmtu) +#else +#define IOCTL_GIFMTU_FUNC_DEF +#endif + +/* esock_ioctl_gifhwaddr */ +#if defined(SIOCGIFHWADDR) && defined(ESOCK_USE_HWADDR) +#define IOCTL_GIFHWADDR_FUNC_DEF IOCTL_GET_FUNC_DEF(gifhwaddr) +#else +#define IOCTL_GIFHWADDR_FUNC_DEF +#endif + +/* esock_ioctl_gifmap */ +#if defined(SIOCGIFMAP) && defined(ESOCK_USE_IFMAP) +#define IOCTL_GIFMAP_FUNC_DEF IOCTL_GET_FUNC_DEF(gifmap) +#else +#define IOCTL_GIFMAP_FUNC_DEF +#endif + +/* esock_ioctl_giftxqlen */ +#if defined(SIOCGIFTXQLEN) +#define IOCTL_GIFTXQLEN_FUNC_DEF IOCTL_GET_FUNC_DEF(giftxqlen) +#else +#define IOCTL_GIFTXQLEN_FUNC_DEF +#endif + +#define IOCTL_GET_FUNCS_DEF \ + IOCTL_GIFINDEX_FUNC_DEF; \ + IOCTL_GIFFLAGS_FUNC_DEF; \ + IOCTL_GIFADDR_FUNC_DEF; \ + IOCTL_GIFDSTADDR_FUNC_DEF; \ + IOCTL_GIFBRDADDR_FUNC_DEF; \ + IOCTL_GIFNETMASK_FUNC_DEF; \ + IOCTL_GIFMTU_FUNC_DEF; \ + IOCTL_GIFHWADDR_FUNC_DEF; \ + IOCTL_GIFMAP_FUNC_DEF; \ + IOCTL_GIFTXQLEN_FUNC_DEF; +#define IOCTL_GET_FUNC_DEF(F) \ + static ERL_NIF_TERM esock_ioctl_##F(ErlNifEnv* env, \ + ESockDescriptor* descP, \ + ERL_NIF_TERM ename) +IOCTL_GET_FUNCS_DEF +#undef IOCTL_GET_FUNC_DEF + +/* esock_ioctl_sifflags */ +#if defined(SIOCSIFFLAGS) +#define IOCTL_SIFFLAGS_FUNC_DEF IOCTL_SET_FUNC_DEF(sifflags) +#else +#define IOCTL_SIFFLAGS_FUNC_DEF +#endif + +/* esock_ioctl_sifaddr */ +#if defined(SIOCSIFADDR) +#define IOCTL_SIFADDR_FUNC_DEF IOCTL_SET_FUNC_DEF(sifaddr) +#else +#define IOCTL_SIFADDR_FUNC_DEF +#endif + +/* esock_ioctl_sifdstaddr */ +#if defined(SIOCSIFDSTADDR) +#define IOCTL_SIFDSTADDR_FUNC_DEF IOCTL_SET_FUNC_DEF(sifdstaddr) +#else +#define IOCTL_SIFDSTADDR_FUNC_DEF +#endif + +/* esock_ioctl_sifbrdaddr */ +#if defined(SIOCSIFBRDADDR) +#define IOCTL_SIFBRDADDR_FUNC_DEF IOCTL_SET_FUNC_DEF(sifbrdaddr) +#else +#define IOCTL_SIFBRDADDR_FUNC_DEF +#endif + +/* esock_ioctl_sifnetmask */ +#if defined(SIOCSIFNETMASK) +#define IOCTL_SIFNETMASK_FUNC_DEF IOCTL_SET_FUNC_DEF(sifnetmask) +#else +#define IOCTL_SIFNETMASK_FUNC_DEF +#endif + +/* esock_ioctl_sifmtu */ +#if defined(SIOCSIFMTU) +#define IOCTL_SIFMTU_FUNC_DEF IOCTL_SET_FUNC_DEF(sifmtu) +#else +#define IOCTL_SIFMTU_FUNC_DEF +#endif + +/* esock_ioctl_siftxqlen */ +#if defined(SIOCSIFTXQLEN) +#define IOCTL_SIFTXQLEN_FUNC_DEF IOCTL_SET_FUNC_DEF(siftxqlen) +#else +#define IOCTL_SIFTXQLEN_FUNC_DEF +#endif + +#define IOCTL_SET_FUNCS_DEF \ + IOCTL_SIFFLAGS_FUNC_DEF; \ + IOCTL_SIFADDR_FUNC_DEF; \ + IOCTL_SIFDSTADDR_FUNC_DEF; \ + IOCTL_SIFBRDADDR_FUNC_DEF; \ + IOCTL_SIFNETMASK_FUNC_DEF; \ + IOCTL_SIFMTU_FUNC_DEF; \ + IOCTL_SIFTXQLEN_FUNC_DEF; +#define IOCTL_SET_FUNC_DEF(F) \ + static ERL_NIF_TERM esock_ioctl_##F(ErlNifEnv* env, \ + ESockDescriptor* descP, \ + ERL_NIF_TERM ename, \ + ERL_NIF_TERM evalue) +IOCTL_SET_FUNCS_DEF +#undef IOCTL_SET_FUNC_DEF + + +static ERL_NIF_TERM encode_ioctl_ifconf(ErlNifEnv* env, + ESockDescriptor* descP, + struct ifconf* ifcP); +static ERL_NIF_TERM encode_ioctl_ifconf_ifreq(ErlNifEnv* env, + ESockDescriptor* descP, + struct ifreq* ifrP); +static ERL_NIF_TERM encode_ioctl_ifreq_name(ErlNifEnv* env, + char* name); +static ERL_NIF_TERM encode_ioctl_ifreq_sockaddr(ErlNifEnv* env, + struct sockaddr* sa); +static ERL_NIF_TERM make_ifreq(ErlNifEnv* env, + ERL_NIF_TERM name, + ERL_NIF_TERM key2, + ERL_NIF_TERM val2); +#if defined(SIOCGIFMAP) && defined(ESOCK_USE_IFMAP) +static ERL_NIF_TERM encode_ioctl_ifrmap(ErlNifEnv* env, + ESockDescriptor* descP, + struct ifmap* mapP); +#endif +#if defined(SIOCGIFHWADDR) && defined(ESOCK_USE_HWADDR) +static ERL_NIF_TERM encode_ioctl_hwaddr(ErlNifEnv* env, + ESockDescriptor* descP, + struct sockaddr* addrP); +#endif +static ERL_NIF_TERM encode_ioctl_ifraddr(ErlNifEnv* env, + ESockDescriptor* descP, + struct sockaddr* addrP); +static ERL_NIF_TERM encode_ioctl_flags(ErlNifEnv* env, + ESockDescriptor* descP, + short flags); +#if defined(SIOCSIFFLAGS) +static BOOLEAN_T decode_ioctl_flags(ErlNifEnv* env, + ESockDescriptor* descP, + ERL_NIF_TERM eflags, + short* flags); +#endif +static BOOLEAN_T decode_ioctl_sockaddr(ErlNifEnv* env, + ESockDescriptor* descP, + ERL_NIF_TERM eaddr, + ESockAddress* addr); +#if defined(SIOCSIFMTU) +static BOOLEAN_T decode_ioctl_mtu(ErlNifEnv* env, + ESockDescriptor* descP, + ERL_NIF_TERM emtu, + int* mtu); +#endif +#if defined(SIOCSIFTXQLEN) +static BOOLEAN_T decode_ioctl_txqlen(ErlNifEnv* env, + ESockDescriptor* descP, + ERL_NIF_TERM etxqlen, + int* txqlen); +#endif +#if defined(SIOCSIFTXQLEN) +static BOOLEAN_T decode_ioctl_ivalue(ErlNifEnv* env, + ESockDescriptor* descP, + ERL_NIF_TERM eivalue, + int* ivalue); +#endif +static ERL_NIF_TERM encode_ioctl_ivalue(ErlNifEnv* env, + ESockDescriptor* descP, + int ivalue); + static ERL_NIF_TERM esock_cancel(ErlNifEnv* env, ESockDescriptor* descP, ERL_NIF_TERM op, @@ -3459,7 +3976,16 @@ static const struct in6_addr in6addr_loopback = GLOBAL_ATOM_DECL(addrform); \ GLOBAL_ATOM_DECL(add_membership); \ GLOBAL_ATOM_DECL(add_source_membership); \ + GLOBAL_ATOM_DECL(allmulti); \ GLOBAL_ATOM_DECL(any); \ + GLOBAL_ATOM_DECL(arphrd_dlci); \ + GLOBAL_ATOM_DECL(arphrd_ether); \ + GLOBAL_ATOM_DECL(arphrd_frelay); \ + GLOBAL_ATOM_DECL(arphrd_ieee802); \ + GLOBAL_ATOM_DECL(arphrd_ieee1394); \ + GLOBAL_ATOM_DECL(arphrd_loopback); \ + GLOBAL_ATOM_DECL(arphrd_netrom); \ + GLOBAL_ATOM_DECL(arphrd_none); \ GLOBAL_ATOM_DECL(associnfo); \ GLOBAL_ATOM_DECL(authhdr); \ GLOBAL_ATOM_DECL(auth_active_key); \ @@ -3469,11 +3995,14 @@ static const struct in6_addr in6addr_loopback = GLOBAL_ATOM_DECL(auth_key); \ GLOBAL_ATOM_DECL(auth_level); \ GLOBAL_ATOM_DECL(autoclose); \ + GLOBAL_ATOM_DECL(automedia); \ GLOBAL_ATOM_DECL(bad_data); \ GLOBAL_ATOM_DECL(bindtodevice); \ GLOBAL_ATOM_DECL(block_source); \ GLOBAL_ATOM_DECL(broadcast); \ GLOBAL_ATOM_DECL(busy_poll); \ + GLOBAL_ATOM_DECL(cantconfig); \ + GLOBAL_ATOM_DECL(chaos); \ GLOBAL_ATOM_DECL(checksum); \ GLOBAL_ATOM_DECL(close); \ GLOBAL_ATOM_DECL(cmsg_cloexec); \ @@ -3496,9 +4025,13 @@ static const struct in6_addr in6addr_loopback = GLOBAL_ATOM_DECL(domain); \ GLOBAL_ATOM_DECL(dontfrag); \ GLOBAL_ATOM_DECL(dontroute); \ + GLOBAL_ATOM_DECL(dormant); \ GLOBAL_ATOM_DECL(drop_membership); \ GLOBAL_ATOM_DECL(drop_source_membership); \ GLOBAL_ATOM_DECL(dstopts); \ + GLOBAL_ATOM_DECL(dying); \ + GLOBAL_ATOM_DECL(dynamic); \ + GLOBAL_ATOM_DECL(echo); \ GLOBAL_ATOM_DECL(egp); \ GLOBAL_ATOM_DECL(enotsup); \ GLOBAL_ATOM_DECL(eor); \ @@ -3527,6 +4060,7 @@ static const struct in6_addr in6addr_loopback = GLOBAL_ATOM_DECL(icmp6); \ GLOBAL_ATOM_DECL(ifindex); \ GLOBAL_ATOM_DECL(igmp); \ + GLOBAL_ATOM_DECL(implink); \ GLOBAL_ATOM_DECL(inet); \ GLOBAL_ATOM_DECL(inet6); \ GLOBAL_ATOM_DECL(info); \ @@ -3545,19 +4079,27 @@ static const struct in6_addr in6addr_loopback = GLOBAL_ATOM_DECL(keepidle); \ GLOBAL_ATOM_DECL(keepintvl); \ GLOBAL_ATOM_DECL(kernel); \ + GLOBAL_ATOM_DECL(knowsepoch); \ GLOBAL_ATOM_DECL(leave_group); \ GLOBAL_ATOM_DECL(level); \ GLOBAL_ATOM_DECL(linger); \ + GLOBAL_ATOM_DECL(link); \ + GLOBAL_ATOM_DECL(link0); \ + GLOBAL_ATOM_DECL(link1); \ + GLOBAL_ATOM_DECL(link2); \ GLOBAL_ATOM_DECL(local); \ GLOBAL_ATOM_DECL(local_auth_chunks); \ - GLOBAL_ATOM_DECL(loopback); \ + GLOBAL_ATOM_DECL(loopback); \ GLOBAL_ATOM_DECL(lowdelay); \ + GLOBAL_ATOM_DECL(lower_up); \ GLOBAL_ATOM_DECL(mark); \ + GLOBAL_ATOM_DECL(master); \ GLOBAL_ATOM_DECL(maxburst); \ GLOBAL_ATOM_DECL(maxseg); \ GLOBAL_ATOM_DECL(md5sig); \ GLOBAL_ATOM_DECL(mincost); \ GLOBAL_ATOM_DECL(minttl); \ + GLOBAL_ATOM_DECL(monitor); \ GLOBAL_ATOM_DECL(more); \ GLOBAL_ATOM_DECL(msfilter); \ GLOBAL_ATOM_DECL(mtu); \ @@ -3568,13 +4110,19 @@ static const struct in6_addr in6addr_loopback = GLOBAL_ATOM_DECL(multicast_if); \ GLOBAL_ATOM_DECL(multicast_loop); \ GLOBAL_ATOM_DECL(multicast_ttl); \ + GLOBAL_ATOM_DECL(name); \ + GLOBAL_ATOM_DECL(noarp); \ GLOBAL_ATOM_DECL(nodelay); \ GLOBAL_ATOM_DECL(nodefrag); \ + GLOBAL_ATOM_DECL(nogroup); \ + GLOBAL_ATOM_DECL(none); \ GLOBAL_ATOM_DECL(noopt); \ GLOBAL_ATOM_DECL(nopush); \ GLOBAL_ATOM_DECL(nosignal); \ + GLOBAL_ATOM_DECL(notrailers); \ GLOBAL_ATOM_DECL(not_found); \ GLOBAL_ATOM_DECL(not_owner); \ + GLOBAL_ATOM_DECL(oactive); \ GLOBAL_ATOM_DECL(ok); \ GLOBAL_ATOM_DECL(oob); \ GLOBAL_ATOM_DECL(oobinline); \ @@ -3594,11 +4142,16 @@ static const struct in6_addr in6addr_loopback = GLOBAL_ATOM_DECL(pktinfo); \ GLOBAL_ATOM_DECL(pktoptions); \ GLOBAL_ATOM_DECL(pkttype); \ + GLOBAL_ATOM_DECL(pointopoint); \ GLOBAL_ATOM_DECL(port); \ GLOBAL_ATOM_DECL(portrange); \ + GLOBAL_ATOM_DECL(portsel); \ + GLOBAL_ATOM_DECL(ppromisc); \ GLOBAL_ATOM_DECL(primary_addr); \ GLOBAL_ATOM_DECL(priority); \ + GLOBAL_ATOM_DECL(promisc); \ GLOBAL_ATOM_DECL(protocol); \ + GLOBAL_ATOM_DECL(pup); \ GLOBAL_ATOM_DECL(raw); \ GLOBAL_ATOM_DECL(rcvbuf); \ GLOBAL_ATOM_DECL(rcvbufforce); \ @@ -3618,7 +4171,8 @@ static const struct in6_addr in6addr_loopback = GLOBAL_ATOM_DECL(recvtclass); \ GLOBAL_ATOM_DECL(recvtos); \ GLOBAL_ATOM_DECL(recvttl); \ - GLOBAL_ATOM_DECL(reliability); \ + GLOBAL_ATOM_DECL(reliability); \ + GLOBAL_ATOM_DECL(renaming); \ GLOBAL_ATOM_DECL(reset_streams); \ GLOBAL_ATOM_DECL(retopts); \ GLOBAL_ATOM_DECL(reuseaddr); \ @@ -3627,6 +4181,7 @@ static const struct in6_addr in6addr_loopback = GLOBAL_ATOM_DECL(router_alert); \ GLOBAL_ATOM_DECL(rthdr); \ GLOBAL_ATOM_DECL(rtoinfo); \ + GLOBAL_ATOM_DECL(running); \ GLOBAL_ATOM_DECL(rxq_ovfl); \ GLOBAL_ATOM_DECL(scope_id); \ GLOBAL_ATOM_DECL(sctp); \ @@ -3640,12 +4195,15 @@ static const struct in6_addr in6addr_loopback = GLOBAL_ATOM_DECL(seqpacket); \ GLOBAL_ATOM_DECL(setfib); \ GLOBAL_ATOM_DECL(set_peer_primary_addr); \ - GLOBAL_ATOM_DECL(socket); \ + GLOBAL_ATOM_DECL(simplex); \ + GLOBAL_ATOM_DECL(slave); \ GLOBAL_ATOM_DECL(sndbuf); \ GLOBAL_ATOM_DECL(sndbufforce); \ GLOBAL_ATOM_DECL(sndlowat); \ GLOBAL_ATOM_DECL(sndtimeo); \ + GLOBAL_ATOM_DECL(socket); \ GLOBAL_ATOM_DECL(spec_dst); \ + GLOBAL_ATOM_DECL(staticarp); \ GLOBAL_ATOM_DECL(status); \ GLOBAL_ATOM_DECL(stream); \ GLOBAL_ATOM_DECL(syncnt); \ @@ -3664,6 +4222,7 @@ static const struct in6_addr in6addr_loopback = GLOBAL_ATOM_DECL(undefined); \ GLOBAL_ATOM_DECL(unicast_hops); \ GLOBAL_ATOM_DECL(unspec); \ + GLOBAL_ATOM_DECL(up); \ GLOBAL_ATOM_DECL(usec); \ GLOBAL_ATOM_DECL(user); \ GLOBAL_ATOM_DECL(user_timeout); \ @@ -3700,6 +4259,7 @@ ERL_NIF_TERM esock_atom_socket_tag; // This has a "special" name ('$socket') LOCAL_ATOM_DECL(association); \ LOCAL_ATOM_DECL(assoc_id); \ LOCAL_ATOM_DECL(authentication); \ + LOCAL_ATOM_DECL(base_addr); \ LOCAL_ATOM_DECL(boolean); \ LOCAL_ATOM_DECL(bound); \ LOCAL_ATOM_DECL(bufsz); \ @@ -3718,6 +4278,7 @@ ERL_NIF_TERM esock_atom_socket_tag; // This has a "special" name ('$socket') LOCAL_ATOM_DECL(debug_filename); \ LOCAL_ATOM_DECL(del); \ LOCAL_ATOM_DECL(dest_unreach); \ + LOCAL_ATOM_DECL(dma); \ LOCAL_ATOM_DECL(do); \ LOCAL_ATOM_DECL(dont); \ LOCAL_ATOM_DECL(dtor); \ @@ -3726,6 +4287,18 @@ ERL_NIF_TERM esock_atom_socket_tag; // This has a "special" name ('$socket') LOCAL_ATOM_DECL(exclude); \ LOCAL_ATOM_DECL(false); \ LOCAL_ATOM_DECL(frag_needed); \ + LOCAL_ATOM_DECL(gifaddr); \ + LOCAL_ATOM_DECL(gifbrdaddr); \ + LOCAL_ATOM_DECL(gifconf); \ + LOCAL_ATOM_DECL(gifdstaddr); \ + LOCAL_ATOM_DECL(gifflags); \ + LOCAL_ATOM_DECL(gifhwaddr); \ + LOCAL_ATOM_DECL(gifindex); \ + LOCAL_ATOM_DECL(gifmap); \ + LOCAL_ATOM_DECL(gifmtu); \ + LOCAL_ATOM_DECL(gifname); \ + LOCAL_ATOM_DECL(gifnetmask); \ + LOCAL_ATOM_DECL(giftxqlen); \ LOCAL_ATOM_DECL(host_unknown); \ LOCAL_ATOM_DECL(host_unreach); \ LOCAL_ATOM_DECL(how); \ @@ -3735,24 +4308,32 @@ ERL_NIF_TERM esock_atom_socket_tag; // This has a "special" name ('$socket') LOCAL_ATOM_DECL(initial); \ LOCAL_ATOM_DECL(interface); \ LOCAL_ATOM_DECL(integer); \ + LOCAL_ATOM_DECL(ioctl_flags); \ + LOCAL_ATOM_DECL(ioctl_requests); \ LOCAL_ATOM_DECL(iov_max); \ LOCAL_ATOM_DECL(iow); \ + LOCAL_ATOM_DECL(irq); \ LOCAL_ATOM_DECL(listening); \ LOCAL_ATOM_DECL(local_rwnd); \ + LOCAL_ATOM_DECL(map); \ LOCAL_ATOM_DECL(max); \ LOCAL_ATOM_DECL(max_attempts); \ LOCAL_ATOM_DECL(max_init_timeo); \ LOCAL_ATOM_DECL(max_instreams); \ LOCAL_ATOM_DECL(asocmaxrxt); \ + LOCAL_ATOM_DECL(mem_end); \ + LOCAL_ATOM_DECL(mem_start); \ LOCAL_ATOM_DECL(min); \ LOCAL_ATOM_DECL(missing); \ LOCAL_ATOM_DECL(mode); \ LOCAL_ATOM_DECL(msg); \ LOCAL_ATOM_DECL(msg_flags); \ + LOCAL_ATOM_DECL(mtu); \ LOCAL_ATOM_DECL(multiaddr); \ LOCAL_ATOM_DECL(net_unknown); \ LOCAL_ATOM_DECL(net_unreach); \ LOCAL_ATOM_DECL(netns); \ + LOCAL_ATOM_DECL(nogroup); \ LOCAL_ATOM_DECL(none); \ LOCAL_ATOM_DECL(noroute); \ LOCAL_ATOM_DECL(not_neighbour); \ @@ -3786,6 +4367,7 @@ ERL_NIF_TERM esock_atom_socket_tag; // This has a "special" name ('$socket') LOCAL_ATOM_DECL(peer_rwnd); \ LOCAL_ATOM_DECL(pkt_toobig); \ LOCAL_ATOM_DECL(policy_fail); \ + LOCAL_ATOM_DECL(port); \ LOCAL_ATOM_DECL(port_unreach); \ LOCAL_ATOM_DECL(prim_file); \ LOCAL_ATOM_DECL(probe); \ @@ -3817,6 +4399,13 @@ ERL_NIF_TERM esock_atom_socket_tag; // This has a "special" name ('$socket') LOCAL_ATOM_DECL(sendfile_tries); \ LOCAL_ATOM_DECL(sendfile_waits); \ LOCAL_ATOM_DECL(shutdown); \ + LOCAL_ATOM_DECL(sifaddr); \ + LOCAL_ATOM_DECL(sifbrdaddr); \ + LOCAL_ATOM_DECL(sifdstaddr); \ + LOCAL_ATOM_DECL(sifflags); \ + LOCAL_ATOM_DECL(sifmtu); \ + LOCAL_ATOM_DECL(sifnetmask); \ + LOCAL_ATOM_DECL(siftxqlen); \ LOCAL_ATOM_DECL(slist); \ LOCAL_ATOM_DECL(sndctrlbuf); \ LOCAL_ATOM_DECL(sockaddr); \ @@ -3828,6 +4417,7 @@ ERL_NIF_TERM esock_atom_socket_tag; // This has a "special" name ('$socket') LOCAL_ATOM_DECL(time_exceeded); \ LOCAL_ATOM_DECL(timeout); \ LOCAL_ATOM_DECL(true); \ + LOCAL_ATOM_DECL(txqlen); \ LOCAL_ATOM_DECL(txstatus); \ LOCAL_ATOM_DECL(txtime); \ LOCAL_ATOM_DECL(use_registry); \ @@ -4737,11 +5327,15 @@ ERL_NIF_TERM esock_supports_1(ErlNifEnv* env, ERL_NIF_TERM key) "\r\n", key) ); if (COMPARE(key, atom_msg_flags) == 0) - result = esock_supports_msg_flags(env); + result = esock_supports_msg_flags(env); else if (COMPARE(key, atom_protocols) == 0) result = esock_supports_protocols(env); + else if (COMPARE(key, atom_ioctl_requests) == 0) + result = esock_supports_ioctl_requests(env); + else if (COMPARE(key, atom_ioctl_flags) == 0) + result = esock_supports_ioctl_flags(env); else if (COMPARE(key, atom_options) == 0) - result = esock_supports_options(env); + result = esock_supports_options(env); else result = MKEL(env); @@ -4754,20 +5348,20 @@ ERL_NIF_TERM esock_supports_1(ErlNifEnv* env, ERL_NIF_TERM key) #ifndef __WIN32__ static ERL_NIF_TERM esock_supports_msg_flags(ErlNifEnv* env) { - size_t n; - ERL_NIF_TERM result; + size_t n; + ERL_NIF_TERM result; - result = MKEL(env); - for (n = 0; n < NUM(msg_flags); n++) { - result = - MKC(env, - MKT2(env, - *(msg_flags[n].name), - MKI(env, msg_flags[n].flag)), - result); - } + result = MKEL(env); + for (n = 0; n < NUM(msg_flags); n++) { + result = + MKC(env, + MKT2(env, + *(msg_flags[n].name), + MKI(env, msg_flags[n].flag)), + result); + } - return result; + return result; } #endif // #ifndef __WIN32__ @@ -4777,92 +5371,206 @@ static ERL_NIF_TERM esock_supports_msg_flags(ErlNifEnv* env) { static ERL_NIF_TERM esock_supports_protocols(ErlNifEnv* env) { - ERL_NIF_TERM protocols; - struct protoent *pe; - int stayopen; + ERL_NIF_TERM protocols; + + protocols = MKEL(env); - protocols = MKEL(env); +#if defined(HAVE_GETPROTOENT) && \ + defined(HAVE_SETPROTOENT) && \ + defined(HAVE_ENDPROTOENT) -#if defined(HAVE_GETPROTOENT) && \ - defined(HAVE_SETPROTOENT) && \ - defined(HAVE_ENDPROTOENT) + { + struct protoent *pe; + int stayopen; MLOCK(data.protocolsMtx); stayopen = TRUE; setprotoent(stayopen); while ((pe = getprotoent()) != NULL) { - ERL_NIF_TERM names; - char **aliases; + ERL_NIF_TERM names; + char **aliases; - names = MKEL(env); - for (aliases = pe->p_aliases; *aliases != NULL; aliases++) - names = MKC(env, MKA(env, *aliases), names); - names = MKC(env, MKA(env, pe->p_name), names); + names = MKEL(env); + for (aliases = pe->p_aliases; *aliases != NULL; aliases++) + names = MKC(env, MKA(env, *aliases), names); + names = MKC(env, MKA(env, pe->p_name), names); - protocols = - MKC(env, MKT2(env, names, MKI(env, pe->p_proto)), protocols); + protocols = + MKC(env, MKT2(env, names, MKI(env, pe->p_proto)), protocols); } endprotoent(); MUNLOCK(data.protocolsMtx); - + } #endif - /* Defaults for known protocols in case getprotoent() - * does not work or does not exist. Prepended to the list - * so a subsequent maps:from_list/2 will take the default - * only when there is nothing from getprotoent(). - */ + /* Defaults for known protocols in case getprotoent() + * does not work or does not exist. Prepended to the list + * so a subsequent maps:from_list/2 will take the default + * only when there is nothing from getprotoent(). + */ - protocols = - MKC(env, - MKT2(env, - MKL1(env, esock_atom_ip), - MKI(env, + protocols = + MKC(env, + MKT2(env, + MKL1(env, esock_atom_ip), + MKI(env, #ifdef SOL_IP - SOL_IP + SOL_IP #else - IPPROTO_IP + IPPROTO_IP #endif - )), - protocols); + )), + protocols); #ifdef HAVE_IPV6 - protocols = - MKC(env, - MKT2(env, - MKL1(env, esock_atom_ipv6), - MKI(env, + protocols = + MKC(env, + MKT2(env, + MKL1(env, esock_atom_ipv6), + MKI(env, #ifdef SOL_IPV6 - SOL_IPV6 + SOL_IPV6 #else - IPPROTO_IPV6 + IPPROTO_IPV6 #endif - )), - protocols); + )), + protocols); #endif - protocols = - MKC(env, - MKT2(env, MKL1(env, esock_atom_tcp), MKI(env, IPPROTO_TCP)), - protocols); + protocols = + MKC(env, + MKT2(env, MKL1(env, esock_atom_tcp), MKI(env, IPPROTO_TCP)), + protocols); - protocols = - MKC(env, - MKT2(env, MKL1(env, esock_atom_udp), MKI(env, IPPROTO_UDP)), - protocols); + protocols = + MKC(env, + MKT2(env, MKL1(env, esock_atom_udp), MKI(env, IPPROTO_UDP)), + protocols); #ifdef HAVE_SCTP - protocols = - MKC(env, - MKT2(env, MKL1(env, esock_atom_sctp), MKI(env, IPPROTO_SCTP)), - protocols); + protocols = + MKC(env, + MKT2(env, MKL1(env, esock_atom_sctp), MKI(env, IPPROTO_SCTP)), + protocols); +#endif + + return protocols; +} +#endif // #ifndef __WIN32__ + + + +#ifndef __WIN32__ +static +ERL_NIF_TERM esock_supports_ioctl_requests(ErlNifEnv* env) +{ + ERL_NIF_TERM requests; + + requests = MKEL(env); + + /* --- GET REQUESTS --- */ +#if defined(SIOCGIFNAME) + requests = MKC(env, MKT2(env, atom_gifname, MKUL(env, SIOCGIFNAME)), requests); +#endif + +#if defined(SIOCGIFINDEX) + requests = MKC(env, MKT2(env, atom_gifindex, MKUL(env, SIOCGIFINDEX)), requests); +#endif + +#if defined(SIOCGIFFLAGS) + requests = MKC(env, MKT2(env, atom_gifflags, MKUL(env, SIOCGIFFLAGS)), requests); +#endif + +#if defined(SIOCGIFADDR) + requests = MKC(env, MKT2(env, atom_gifaddr, MKUL(env, SIOCGIFADDR)), requests); +#endif + +#if defined(SIOCGIFDSTADDR) + requests = MKC(env, MKT2(env, atom_gifdstaddr, MKUL(env, SIOCGIFDSTADDR)), requests); +#endif + +#if defined(SIOCGIFBRDADDR) + requests = MKC(env, MKT2(env, atom_gifbrdaddr, MKUL(env, SIOCGIFBRDADDR)), requests); #endif - return protocols; +#if defined(SIOCGIFNETMASK) + requests = MKC(env, MKT2(env, atom_gifnetmask, MKUL(env, SIOCGIFNETMASK)), requests); +#endif + +#if defined(SIOCGIFMTU) + requests = MKC(env, MKT2(env, atom_gifmtu, MKUL(env, SIOCGIFMTU)), requests); +#endif + +#if defined(SIOCGIFHWADDR) && defined(ESOCK_USE_HWADDR) + requests = MKC(env, MKT2(env, atom_gifhwaddr, MKUL(env, SIOCGIFHWADDR)), requests); +#endif + +#if defined(SIOCGIFMAP) && defined(ESOCK_USE_IFMAP) + requests = MKC(env, MKT2(env, atom_gifmap, MKUL(env, SIOCGIFMAP)), requests); +#endif + +#if defined(SIOCGIFTXQLEN) + requests = MKC(env, MKT2(env, atom_giftxqlen, MKUL(env, SIOCGIFTXQLEN)), requests); +#endif + +#if defined(SIOCGIFCONF) + requests = MKC(env, MKT2(env, atom_gifconf, MKUL(env, SIOCGIFCONF)), requests); +#endif + + /* --- SET REQUESTS --- */ +#if defined(SIOCSIFFLAGS) + requests = MKC(env, MKT2(env, atom_sifflags, MKUL(env, SIOCSIFFLAGS)), requests); +#endif + +#if defined(SIOCSIFADDR) + requests = MKC(env, MKT2(env, atom_sifaddr, MKUL(env, SIOCSIFADDR)), requests); +#endif + +#if defined(SIOCSIFDSTADDR) + requests = MKC(env, MKT2(env, atom_sifdstaddr, MKUL(env, SIOCSIFDSTADDR)), requests); +#endif + +#if defined(SIOCSIFBRDADDR) + requests = MKC(env, MKT2(env, atom_sifbrdaddr, MKUL(env, SIOCSIFBRDADDR)), requests); +#endif + +#if defined(SIOCSIFMTU) + requests = MKC(env, MKT2(env, atom_sifmtu, MKUL(env, SIOCSIFMTU)), requests); +#endif + +#if defined(SIOCSIFTXQLEN) + requests = MKC(env, MKT2(env, atom_siftxqlen, MKUL(env, SIOCSIFTXQLEN)), requests); +#endif + + return requests; } #endif // #ifndef __WIN32__ +#ifndef __WIN32__ + +static ERL_NIF_TERM esock_supports_ioctl_flags(ErlNifEnv* env) +{ + size_t n; + ERL_NIF_TERM result; + + result = MKEL(env); + for (n = 0; n < NUM(ioctl_flags); n++) { + result = + MKC(env, + MKT2(env, + *(ioctl_flags[n].name), + MKI(env, ioctl_flags[n].flag)), + result); + } + + return result; +} + +#endif // #ifndef __WIN32__ + + + #ifndef __WIN32__ @@ -5549,7 +6257,7 @@ ERL_NIF_TERM nif_bind(ErlNifEnv* env, ESockDescriptor* descP; ERL_NIF_TERM eSockAddr, ret; ESockAddress sockAddr; - SOCKLEN_T addrLen; + SOCKLEN_T addrLen; ESOCK_ASSERT( argc == 2 ); @@ -12365,36 +13073,36 @@ ERL_NIF_TERM nif_peername(ErlNifEnv* env, const ERL_NIF_TERM argv[]) { #ifdef __WIN32__ - return enif_raise_exception(env, MKA(env, "notsup")); + return enif_raise_exception(env, MKA(env, "notsup")); #else - ESockDescriptor* descP; - ERL_NIF_TERM res; + ESockDescriptor* descP; + ERL_NIF_TERM res; - ESOCK_ASSERT( argc == 1 ); + ESOCK_ASSERT( argc == 1 ); - SGDBG( ("SOCKET", "nif_peername -> entry with argc: %d\r\n", argc) ); + SGDBG( ("SOCKET", "nif_peername -> entry with argc: %d\r\n", argc) ); - /* Extract arguments and perform preliminary validation */ + /* Extract arguments and perform preliminary validation */ - if (! ESOCK_GET_RESOURCE(env, argv[0], (void**) &descP)) { - return enif_make_badarg(env); - } + if (! ESOCK_GET_RESOURCE(env, argv[0], (void**) &descP)) { + return enif_make_badarg(env); + } - MLOCK(descP->readMtx); + MLOCK(descP->readMtx); - SSDBG( descP, - ("SOCKET", "nif_peername(%T) {%d}" - "\r\n", argv[0], descP->sock) ); + SSDBG( descP, + ("SOCKET", "nif_peername(%T) {%d}" + "\r\n", argv[0], descP->sock) ); - res = esock_peername(env, descP); + res = esock_peername(env, descP); - SSDBG( descP, - ("SOCKET", "nif_peername(%T) {%d} -> done with res = %T\r\n", - argv[0], descP->sock, res) ); + SSDBG( descP, + ("SOCKET", "nif_peername(%T) {%d} -> done with res = %T\r\n", + argv[0], descP->sock, res) ); - MUNLOCK(descP->readMtx); + MUNLOCK(descP->readMtx); - return res; + return res; #endif // #ifdef __WIN32__ #else } @@ -12405,23 +13113,1252 @@ static ERL_NIF_TERM esock_peername(ErlNifEnv* env, ESockDescriptor* descP) { - ESockAddress sa; - ESockAddress* saP = &sa; - SOCKLEN_T sz = sizeof(ESockAddress); + ESockAddress sa; + ESockAddress* saP = &sa; + SOCKLEN_T sz = sizeof(ESockAddress); - if (! IS_OPEN(descP->readState)) - return esock_make_error(env, atom_closed); + if (! IS_OPEN(descP->readState)) + return esock_make_error(env, atom_closed); + + sys_memzero((char*) saP, sz); + if (sock_peer(descP->sock, (struct sockaddr*) saP, &sz) < 0) { + return esock_make_error_errno(env, sock_errno()); + } else { + ERL_NIF_TERM esa; + + esock_encode_sockaddr(env, saP, sz, &esa); + return esock_make_ok2(env, esa); + } +} +#endif // #ifndef __WIN32__ + + + +/* ---------------------------------------------------------------------- + * nif_ioctl - control device - get + * + * Description: + * Returns whatever info the ioctl returns for the specific (get) request. + * WHEN SET IS IMPLEMENTED, WE NED ANOTHER ARGUMENT!! + * + * Arguments: + * Socket (ref) - Points to the socket descriptor. + * Request - The ioctl get/set request + * NameOrIdx - Name or Index of the interface - only for some requests + * Currently only one (get) request does not provide this + * (and the value) argument; gifconf + * Currently, only one (get) request use index; gifname + * All other requests (get and set) use Name as "key". + * Val - Value to *set* + * This argument is *only* provided for set requests. + */ + +static +ERL_NIF_TERM nif_ioctl(ErlNifEnv* env, + int argc, + const ERL_NIF_TERM argv[]) +{ +#ifdef __WIN32__ + return enif_raise_exception(env, MKA(env, "notsup")); +#else + ESockDescriptor* descP; + ERL_NIF_TERM res; + unsigned long req; + + SGDBG( ("SOCKET", "nif_ioctl -> entry with argc: %d\r\n", argc) ); + + ESOCK_ASSERT( (argc == 2) || (argc == 3) || (argc == 4) ); + + if (! ESOCK_GET_RESOURCE(env, argv[0], (void**) &descP)) { + SGDBG( ("SOCKET", "nif_ioctl -> no resource\r\n") ); + return enif_make_badarg(env); + } + + if (! GET_ULONG(env, argv[1], &req)) { + SGDBG( ("SOCKET", "nif_ioctl -> 'request' not 'unsigned long'\r\n") ); + return enif_make_badarg(env); + } + + SSDBG( descP, + ("SOCKET", "nif_ioctl(%T) {%d} -> ioctl request %d" + "\r\n", argv[0], descP->sock, req) ); + + MLOCK(descP->readMtx); + + if (! IS_OPEN(descP->readState)) { + res = esock_make_error(env, atom_closed); + } else { + + if (argc == 2) { + + /* Only one request with this number of arguments: gifconf + * Socket and request (=gifconf) + */ + + /* Two arguments: Socket and get request */ + res = esock_ioctl1(env, descP, req); + + } else if (argc == 3) { + + /* (Currently) All *other* get requests has 3 arguments + * Socket, request and name/index + */ + + ERL_NIF_TERM earg = argv[2]; + + /* Two arguments: request and arg */ + res = esock_ioctl2(env, descP, req, earg); + + } else if (argc == 4) { + + /* (Currently) Set requests has 4 arguments + * Socket, request, name and value + */ + + ERL_NIF_TERM earg1 = argv[2]; // (currently) Name + ERL_NIF_TERM earg2 = argv[3]; // Value + + /* Three arguments: request, arg1 (name) and arg2 (value) */ + res = esock_ioctl3(env, descP, req, earg1, earg2); - sys_memzero((char*) saP, sz); - if (sock_peer(descP->sock, (struct sockaddr*) saP, &sz) < 0) { - return esock_make_error_errno(env, sock_errno()); } else { - ERL_NIF_TERM esa; - esock_encode_sockaddr(env, saP, sz, &esa); - return esock_make_ok2(env, esa); + res = esock_make_error(env, esock_atom_einval); + + } + } + + MUNLOCK(descP->readMtx); + + SSDBG( descP, + ("SOCKET", "nif_ioctl(%T) {%d} -> done with res = %T\r\n", + argv[0], descP->sock, res) ); + + return res; +#endif // #ifdef __WIN32__ #else +} + + + +#ifndef __WIN32__ + +static +ERL_NIF_TERM esock_ioctl1(ErlNifEnv* env, + ESockDescriptor* descP, + unsigned long req) +{ + switch (req) { + +#if defined(SIOCGIFCONF) + case SIOCGIFCONF: + return esock_ioctl_gifconf(env, descP); + break; +#endif + + default: + return esock_make_error(env, esock_atom_enotsup); + break; + } + +} + + +/* The type and value of 'arg' depend on the request, + * which we have not yet "analyzed". + * + * Request arg arg type + * ------- ------- -------- + * gifname ifindex integer + * gifindex name string + * gifflags name string + * gifaddr name string + * gifdstaddr name string + * gifbdraddr name string + * gifnetmask name string + * gifmtu name string + * gifhwaddr name string + * gifmap name string + * giftxqlen name string + */ +static +ERL_NIF_TERM esock_ioctl2(ErlNifEnv* env, + ESockDescriptor* descP, + unsigned long req, + ERL_NIF_TERM arg) +{ + /* This for *get* requests */ + + switch (req) { + +#if defined(SIOCGIFNAME) + case SIOCGIFNAME: + return esock_ioctl_gifname(env, descP, arg); + break; +#endif + +#if defined(SIOCGIFINDEX) + case SIOCGIFINDEX: + return esock_ioctl_gifindex(env, descP, arg); + break; +#endif + +#if defined(SIOCGIFFLAGS) + case SIOCGIFFLAGS: + return esock_ioctl_gifflags(env, descP, arg); + break; +#endif + +#if defined(SIOCGIFADDR) + case SIOCGIFADDR: + return esock_ioctl_gifaddr(env, descP, arg); + break; +#endif + +#if defined(SIOCGIFDSTADDR) + case SIOCGIFDSTADDR: + return esock_ioctl_gifdstaddr(env, descP, arg); + break; +#endif + +#if defined(SIOCGIFBRDADDR) + case SIOCGIFBRDADDR: + return esock_ioctl_gifbrdaddr(env, descP, arg); + break; +#endif + +#if defined(SIOCGIFNETMASK) + case SIOCGIFNETMASK: + return esock_ioctl_gifnetmask(env, descP, arg); + break; +#endif + +#if defined(SIOCGIFMTU) + case SIOCGIFMTU: + return esock_ioctl_gifmtu(env, descP, arg); + break; +#endif + +#if defined(SIOCGIFHWADDR) && defined(ESOCK_USE_HWADDR) + case SIOCGIFHWADDR: + return esock_ioctl_gifhwaddr(env, descP, arg); + break; +#endif + +#if defined(SIOCGIFMAP) && defined(ESOCK_USE_IFMAP) + case SIOCGIFMAP: + return esock_ioctl_gifmap(env, descP, arg); + break; +#endif + +#if defined(SIOCGIFTXQLEN) + case SIOCGIFTXQLEN: + return esock_ioctl_giftxqlen(env, descP, arg); + break; +#endif + + default: + return esock_make_error(env, esock_atom_enotsup); + break; + } + +} + + +/* The type and value of arg(s) depend on the request, + * which we have not yet "analyzed". + * + * Request arg1 arg1 type arg2 arg2 type + * ------- ------- --------- ------ --------- + * sifflags name string Flags #{IntFlag := boolean()} + * IntFlag is the native flag + * sifaddr name string Addr sockaddr() + * sifdstaddr name string DstAddr sockaddr() + * sifbrdaddr name string BrdAddr sockaddr() + * sifnetmask name string NetMask sockaddr() + * gifmtu name string MTU integer() + * sifhwaddr name string HwAddr sockaddr() + * giftxqlen name string Len integer() + */ +static +ERL_NIF_TERM esock_ioctl3(ErlNifEnv* env, + ESockDescriptor* descP, + unsigned long req, + ERL_NIF_TERM ename, + ERL_NIF_TERM eval) +{ + + switch (req) { + +#if defined(SIOCSIFFLAGS) + case SIOCSIFFLAGS: + return esock_ioctl_sifflags(env, descP, ename, eval); + break; +#endif + +#if defined(SIOCSIFADDR) + case SIOCSIFADDR: + return esock_ioctl_sifaddr(env, descP, ename, eval); + break; +#endif + +#if defined(SIOCSIFDSTADDR) + case SIOCSIFDSTADDR: + return esock_ioctl_sifdstaddr(env, descP, ename, eval); + break; +#endif + +#if defined(SIOCSIFBRDADDR) + case SIOCSIFBRDADDR: + return esock_ioctl_sifbrdaddr(env, descP, ename, eval); + break; +#endif + +#if defined(SIOCSIFNETMASK) + case SIOCSIFNETMASK: + return esock_ioctl_sifnetmask(env, descP, ename, eval); + break; +#endif + +#if defined(SIOCSIFMTU) + case SIOCSIFMTU: + return esock_ioctl_sifmtu(env, descP, ename, eval); + break; +#endif + +#if defined(SIOCSIFTXQLEN) + case SIOCSIFTXQLEN: + return esock_ioctl_siftxqlen(env, descP, ename, eval); + break; +#endif + + default: + return esock_make_error(env, esock_atom_enotsup); + break; + } + +} + + +/* =========================================================================== + * The implemented (ioctl) get requests falls into three grops: + * + * 1) gifconf - Takes no argument other then the request + * 2) gifname - Takes the interface index (integer) as an argument + * 3) other - All other (get) requests takes the interface name (string) + * as the argument. + * + * The functions defined using the macros below are all in the third (3) + * group. + * + */ + +/* *** esock_ioctl_gifindex *** */ +#if defined(SIOCGIFINDEX) +#if defined(ESOCK_USE_IFINDEX) +#define IOCTL_GIFINDEX_FUNC_DECL \ + IOCTL_GET_REQUEST_DECL(gifindex, SIOCGIFINDEX, ivalue, ifreq.ifr_ifindex) +#elif defined(ESOCK_USE_INDEX) +#define IOCTL_GIFINDEX_FUNC_DECL \ + IOCTL_GET_REQUEST_DECL(gifindex, SIOCGIFINDEX, ivalue, ifreq.ifr_index) +#else +#define IOCTL_GIFINDEX_FUNC_DECL +#endif +#else +#define IOCTL_GIFINDEX_FUNC_DECL +#endif + +/* *** esock_ioctl_gifflags *** */ +#if defined(SIOCGIFFLAGS) +#define IOCTL_GIFFLAGS_FUNC_DECL \ + IOCTL_GET_REQUEST_DECL(gifflags, SIOCGIFFLAGS, flags, ifreq.ifr_flags) +#else +#define IOCTL_GIFFLAGS_FUNC_DECL +#endif + +/* *** esock_ioctl_gifaddr *** */ +#if defined(SIOCGIFADDR) +#define IOCTL_GIFADDR_FUNC_DECL \ + IOCTL_GET_REQUEST_DECL(gifaddr, SIOCGIFADDR, ifraddr, &ifreq.ifr_addr) +#else +#define IOCTL_GIFADDR_FUNC_DECL +#endif + +/* *** esock_ioctl_gifdstaddr *** */ +#if defined(SIOCGIFDSTADDR) +#define IOCTL_GIFDSTADDR_FUNC_DECL \ + IOCTL_GET_REQUEST_DECL(gifdstaddr, SIOCGIFDSTADDR, ifraddr, &ifreq.ifr_dstaddr) +#else +#define IOCTL_GIFDSTADDR_FUNC_DECL +#endif + +/* *** esock_ioctl_gifbrdaddr *** */ +#if defined(SIOCGIFBRDADDR) +#define IOCTL_GIFBRDADDR_FUNC_DECL \ + IOCTL_GET_REQUEST_DECL(gifbrdaddr, SIOCGIFBRDADDR, ifraddr, &ifreq.ifr_broadaddr) +#else +#define IOCTL_GIFBRDADDR_FUNC_DECL +#endif + +/* *** esock_ioctl_gifnetmask *** */ +#if defined(SIOCGIFNETMASK) +#ifdef __linux__ +#define IOCTL_GIFNETMASK_FUNC_DECL \ + IOCTL_GET_REQUEST_DECL(gifnetmask, SIOCGIFNETMASK, ifraddr, &ifreq.ifr_netmask) +#else +#define IOCTL_GIFNETMASK_FUNC_DECL \ + IOCTL_GET_REQUEST_DECL(gifnetmask, SIOCGIFNETMASK, ifraddr, &ifreq.ifr_addr) +#endif +#else +#define IOCTL_GIFNETMASK_FUNC_DECL +#endif + +/* *** esock_ioctl_gifmtu *** */ +#if defined(SIOCGIFMTU) +#define IOCTL_GIFMTU_FUNC_DECL \ + IOCTL_GET_REQUEST_DECL(gifmtu, SIOCGIFMTU, ivalue, ifreq.ifr_mtu) +#else +#define IOCTL_GIFMTU_FUNC_DECL +#endif + +/* *** esock_ioctl_gifhwaddr *** */ +#if defined(SIOCGIFHWADDR) && defined(ESOCK_USE_HWADDR) +#define IOCTL_GIFHWADDR_FUNC_DECL \ + IOCTL_GET_REQUEST_DECL(gifhwaddr, SIOCGIFHWADDR, hwaddr, &ifreq.ifr_hwaddr) +#else +#define IOCTL_GIFHWADDR_FUNC_DECL +#endif + +/* *** esock_ioctl_gifmap *** */ +#if defined(SIOCGIFMAP) && defined(ESOCK_USE_IFMAP) +#define IOCTL_GIFMAP_FUNC_DECL \ + IOCTL_GET_REQUEST_DECL(gifmap, SIOCGIFMAP, ifrmap, &ifreq.ifr_map) +#else +#define IOCTL_GIFMAP_FUNC_DECL +#endif + +/* *** esock_ioctl_giftxqlen *** */ +#if defined(SIOCGIFTXQLEN) +#define IOCTL_GIFTXQLEN_FUNC_DECL \ + IOCTL_GET_REQUEST_DECL(giftxqlen, SIOCGIFTXQLEN, ivalue, ifreq.ifr_qlen) +#else +#define IOCTL_GIFTXQLEN_FUNC_DECL +#endif + +#define IOCTL_GET_FUNCS \ + IOCTL_GIFINDEX_FUNC_DECL \ + IOCTL_GIFFLAGS_FUNC_DECL \ + IOCTL_GIFADDR_FUNC_DECL \ + IOCTL_GIFDSTADDR_FUNC_DECL \ + IOCTL_GIFBRDADDR_FUNC_DECL \ + IOCTL_GIFNETMASK_FUNC_DECL \ + IOCTL_GIFMTU_FUNC_DECL \ + IOCTL_GIFHWADDR_FUNC_DECL \ + IOCTL_GIFMAP_FUNC_DECL \ + IOCTL_GIFTXQLEN_FUNC_DECL + +#define IOCTL_GET_REQUEST_DECL(OR, R, EF, UV) \ + static \ + ERL_NIF_TERM esock_ioctl_##OR(ErlNifEnv* env, \ + ESockDescriptor* descP, \ + ERL_NIF_TERM ename) \ + { \ + ERL_NIF_TERM result; \ + struct ifreq ifreq; \ + char* ifn = NULL; \ + int nlen; \ + \ + SSDBG( descP, ("SOCKET", "esock_ioctl_" #OR " {%d} -> entry with" \ + "\r\n (e)Name: %T" \ + "\r\n", descP->sock, ename) ); \ + \ + if (!esock_decode_string(env, ename, &ifn)) \ + return enif_make_badarg(env); \ + \ + nlen = esock_strnlen(ifn, IFNAMSIZ); \ + \ + sys_memset(ifreq.ifr_name, '\0', IFNAMSIZ); \ + sys_memcpy(ifreq.ifr_name, ifn, \ + (nlen >= IFNAMSIZ) ? IFNAMSIZ-1 : nlen); \ + \ + SSDBG( descP, \ + ("SOCKET", \ + "esock_ioctl_" #OR " {%d} -> try ioctl\r\n", \ + descP->sock) ); \ + \ + if (ioctl(descP->sock, R, (char *) &ifreq) < 0) { \ + int saveErrno = sock_errno(); \ + ERL_NIF_TERM reason = MKA(env, erl_errno_id(saveErrno)); \ + \ + SSDBG( descP, \ + ("SOCKET", "esock_ioctl_" #OR " {%d} -> failure: " \ + "\r\n reason: %T (%d)" \ + "\r\n", descP->sock, reason, saveErrno) ); \ + \ + result = esock_make_error(env, reason); \ + \ + } else { \ + SSDBG( descP, \ + ("SOCKET", "esock_ioctl_" #OR " {%d} -> encode value\r\n", \ + descP->sock) ); \ + result = encode_ioctl_##EF(env, descP, UV); \ + } \ + \ + FREE(ifn); \ + \ + return result; \ + \ + } +IOCTL_GET_FUNCS +#undef IOCTL_GET_FUNCS + + +/* =========================================================================== + * The "rest" of the implemented (ioctl) get requests + * + * These (get) requests could not be 'generated' by the macros above. + */ + +static +ERL_NIF_TERM esock_ioctl_gifconf(ErlNifEnv* env, + ESockDescriptor* descP) +{ + struct ifconf ifc; + int ifc_len = 0; + int buflen = 100 * sizeof(struct ifreq); + char *buf = MALLOC(buflen); + ERL_NIF_TERM result; + + SSDBG( descP, ("SOCKET", "esock_ioctl_gifconf {%d} -> entry\r\n", descP->sock) ); + + for (;;) { + ifc.ifc_len = buflen; + ifc.ifc_buf = buf; + if (ioctl(descP->sock, SIOCGIFCONF, (char *) &ifc) < 0) { + int saveErrno = sock_errno(); + + SSDBG( descP, + ("SOCKET", "esock_ioctl_gifconf {%d} -> failure: " + "\r\n errno: %d (%s)" + "\r\n", descP->sock, saveErrno, erl_errno_id(saveErrno)) ); + + if (saveErrno != EINVAL || ifc_len) { + ERL_NIF_TERM reason = MKA(env, erl_errno_id(saveErrno)); + FREE(buf); + return esock_make_error(env, reason); + } + } else { + if (ifc.ifc_len == ifc_len) break; /* buf large enough */ + ifc_len = ifc.ifc_len; } + buflen += 10 * sizeof(struct ifreq); + buf = (char *) REALLOC(buf, buflen); + } + + result = encode_ioctl_ifconf(env, descP, &ifc); + + FREE(ifc.ifc_buf); + + return result; } + + +#if defined(SIOCGIFNAME) +static +ERL_NIF_TERM esock_ioctl_gifname(ErlNifEnv* env, + ESockDescriptor* descP, + ERL_NIF_TERM eidx) +{ + ERL_NIF_TERM result; + struct ifreq ifreq; + int index; + + SSDBG( descP, ("SOCKET", "esock_ioctl_gifname {%d} -> entry with" + "\r\n (e)Index: %T" + "\r\n", descP->sock, eidx) ); + + if (!GET_INT(env, eidx, &index)) + return enif_make_badarg(env); + + ifreq.ifr_ifindex = index; + + SSDBG( descP, + ("SOCKET", "esock_ioctl_gifname {%d} -> try ioctl\r\n", descP->sock) ); + + if (ioctl(descP->sock, SIOCGIFNAME, (char *) &ifreq) < 0) { + int saveErrno = sock_errno(); + ERL_NIF_TERM reason = MKA(env, erl_errno_id(saveErrno)); + + SSDBG( descP, + ("SOCKET", "esock_ioctl_gifname {%d} -> failure: " + "\r\n reason: %T (%d)" + "\r\n", descP->sock, reason, saveErrno) ); + + result = esock_make_error(env, reason); + + } else { + SSDBG( descP, + ("SOCKET", "esock_ioctl_gifname {%d} -> encode name\r\n", + descP->sock) ); + + result = esock_make_ok2(env, encode_ioctl_ifreq_name(env, ifreq.ifr_name)); + } + + SSDBG( descP, + ("SOCKET", "esock_ioctl_gifname {%d} -> done with" + "\r\n result: %T" + "\r\n", + descP->sock, result) ); + + return result; + +} +#endif + + + + +/* =========================================================================== + * The implemented (ioctl) set requests: + * + */ + +/* *** esock_ioctl_sifaddr *** */ +#if defined(SIOCSIFADDR) +#define IOCTL_SIFADDR_FUNC_DECL \ + IOCTL_SET_REQUEST_DECL(sifaddr, SIOCSIFADDR, sockaddr, \ + ((ESockAddress*) &ifreq.ifr_addr)) +#else +#define IOCTL_SIFADDR_FUNC_DECL +#endif + +/* *** esock_ioctl_sifdstaddr *** */ +#if defined(SIOCSIFDSTADDR) +#define IOCTL_SIFDSTADDR_FUNC_DECL \ + IOCTL_SET_REQUEST_DECL(sifdstaddr, SIOCSIFDSTADDR, sockaddr, \ + ((ESockAddress*) &ifreq.ifr_dstaddr)) +#else +#define IOCTL_SIFDSTADDR_FUNC_DECL +#endif + +/* *** esock_ioctl_sifbrdaddr *** */ +#if defined(SIOCSIFBRDADDR) +#define IOCTL_SIFBRDADDR_FUNC_DECL \ + IOCTL_SET_REQUEST_DECL(sifbrdaddr, SIOCSIFBRDADDR, sockaddr, \ + ((ESockAddress*) &ifreq.ifr_broadaddr)) +#else +#define IOCTL_SIFBRDADDR_FUNC_DECL +#endif + +/* *** esock_ioctl_sifnetmask *** */ +#if defined(SIOCSIFNETMASK) +#ifdef __linux__ +#define IOCTL_SIFNETMASK_FUNC_DECL \ + IOCTL_SET_REQUEST_DECL(sifnetmask, SIOCSIFNETMASK, sockaddr, \ + ((ESockAddress*) &ifreq.ifr_netmask)) +#else +#define IOCTL_SIFNETMASK_FUNC_DECL \ + IOCTL_SET_REQUEST_DECL(sifnetmask, SIOCSIFNETMASK, sockaddr, \ + ((ESockAddress*) &ifreq.ifr_addr)) +#endif +#else +#define IOCTL_SIFNETMASK_FUNC_DECL +#endif + +/* *** esock_ioctl_sifmtu *** + * On some platforms, MTU is an unsigned int + */ +#if defined(SIOCSIFMTU) +#define IOCTL_SIFMTU_FUNC_DECL \ + IOCTL_SET_REQUEST_DECL(sifmtu, SIOCSIFMTU, mtu, (int*) &ifreq.ifr_mtu) +#else +#define IOCTL_SIFMTU_FUNC_DECL +#endif + +/* *** esock_ioctl_siftxqlen *** */ +#if defined(SIOCSIFTXQLEN) +#define IOCTL_SIFTXQLEN_FUNC_DECL \ + IOCTL_SET_REQUEST_DECL(siftxqlen, SIOCSIFTXQLEN, txqlen, &ifreq.ifr_qlen) +#else +#define IOCTL_SIFTXQLEN_FUNC_DECL +#endif + +#define IOCTL_SET_FUNCS \ + IOCTL_SIFADDR_FUNC_DECL \ + IOCTL_SIFDSTADDR_FUNC_DECL \ + IOCTL_SIFBRDADDR_FUNC_DECL \ + IOCTL_SIFNETMASK_FUNC_DECL \ + IOCTL_SIFMTU_FUNC_DECL \ + IOCTL_SIFTXQLEN_FUNC_DECL + +#define IOCTL_SET_REQUEST_DECL(OR, R, DF, UVP) \ + static \ + ERL_NIF_TERM esock_ioctl_##OR(ErlNifEnv* env, \ + ESockDescriptor* descP, \ + ERL_NIF_TERM ename, \ + ERL_NIF_TERM evalue) \ + { \ + ERL_NIF_TERM result; \ + struct ifreq ifreq; \ + char* ifn = NULL; \ + int nlen; \ + \ + SSDBG( descP, ("SOCKET", "esock_ioctl_" #OR " {%d} -> entry with" \ + "\r\n (e)Name: %T" \ + "\r\n (e)Value: %T" \ + "\r\n", descP->sock, ename, evalue) ); \ + \ + if (!esock_decode_string(env, ename, &ifn)) { \ + \ + SSDBG( descP, \ + ("SOCKET", "esock_ioctl_" #OR " {%d} -> failed decode name" \ + "\r\n", descP->sock) ); \ + \ + return enif_make_badarg(env); \ + } \ + \ + if (! decode_ioctl_##DF(env, descP, evalue, UVP)) { \ + \ + SSDBG( descP, \ + ("SOCKET", "esock_ioctl_" #OR " {%d} -> failed decode addr" \ + "\r\n", descP->sock) ); \ + \ + return esock_make_invalid(env, atom_##DF); \ + } \ + \ + nlen = esock_strnlen(ifn, IFNAMSIZ); \ + \ + sys_memset(ifreq.ifr_name, '\0', IFNAMSIZ); \ + sys_memcpy(ifreq.ifr_name, ifn, \ + (nlen >= IFNAMSIZ) ? IFNAMSIZ-1 : nlen); \ + \ + SSDBG( descP, \ + ("SOCKET", "esock_ioctl_" #OR " {%d} -> try ioctl\r\n", \ + descP->sock) ); \ + \ + if (ioctl(descP->sock, R, (char *) &ifreq) < 0) { \ + int saveErrno = sock_errno(); \ + ERL_NIF_TERM reason = MKA(env, erl_errno_id(saveErrno)); \ + \ + SSDBG( descP, \ + ("SOCKET", "esock_ioctl_" #OR " {%d} -> failure: " \ + "\r\n reason: %T (%d)" \ + "\r\n", descP->sock, reason, saveErrno) ); \ + \ + result = esock_make_error(env, reason); \ + \ + } else { \ + SSDBG( descP, \ + ("SOCKET", "esock_ioctl_" #OR " {%d} -> " \ + "addr successfully set\r\n", \ + descP->sock) ); \ + result = esock_atom_ok; \ + } \ + \ + FREE(ifn); \ + \ + return result; \ + \ + } + +IOCTL_SET_FUNCS +#undef IOCTL_SET_FUNCS + + +/* =========================================================================== + * The "rest" of the implemented (ioctl) set requests + * + * These (set) requests could not be 'generated' by the macros above. + */ + +#if defined(SIOCSIFFLAGS) +static +ERL_NIF_TERM esock_ioctl_sifflags(ErlNifEnv* env, + ESockDescriptor* descP, + ERL_NIF_TERM ename, + ERL_NIF_TERM eflags) +{ + ERL_NIF_TERM result; + struct ifreq ifreq; + char* ifn = NULL; + int nlen; + + SSDBG( descP, ("SOCKET", "esock_ioctl_sifflags {%d} -> entry with" + "\r\n (e)Name: %T" + "\r\n (e)Flags: %T" + "\r\n", descP->sock, ename, eflags) ); + + if (!esock_decode_string(env, ename, &ifn)) { + + SSDBG( descP, + ("SOCKET", "esock_ioctl_sifflags {%d} -> failed decode name" + "\r\n", descP->sock) ); + + return enif_make_badarg(env); + } + + // Make sure the length of the string is valid! + nlen = esock_strnlen(ifn, IFNAMSIZ); + + sys_memset(ifreq.ifr_name, '\0', IFNAMSIZ); // Just in case + sys_memcpy(ifreq.ifr_name, ifn, + (nlen >= IFNAMSIZ) ? IFNAMSIZ-1 : nlen); + + SSDBG( descP, + ("SOCKET", "esock_ioctl_sifflags {%d} -> try (get) ioctl\r\n", + descP->sock) ); + + if (ioctl(descP->sock, SIOCGIFFLAGS, (char *) &ifreq) < 0) { + int saveErrno = sock_errno(); + ERL_NIF_TERM reason = MKA(env, erl_errno_id(saveErrno)); + + SSDBG( descP, + ("SOCKET", "esock_ioctl_sifflags {%d} -> " + "failure: failed reading *current* flags" + "\r\n reason: %T (%d)" + "\r\n", descP->sock, reason, saveErrno) ); + + result = esock_make_error(env, reason); + + } else { + + SSDBG( descP, + ("SOCKET", "esock_ioctl_sifflags {%d} -> (local) update flags\r\n", + descP->sock) ); + + if (decode_ioctl_flags(env, descP, eflags, &ifreq.ifr_flags)) { + + SSDBG( descP, + ("SOCKET", "esock_ioctl_sifflags {%d} -> try (set) ioctl\r\n", + descP->sock) ); + + if (ioctl(descP->sock, SIOCSIFFLAGS, (char *) &ifreq) < 0) { + int saveErrno = sock_errno(); + ERL_NIF_TERM reason = MKA(env, erl_errno_id(saveErrno)); + + SSDBG( descP, + ("SOCKET", "esock_ioctl_sifflags {%d} -> failure: " + "\r\n reason: %T (%d)" + "\r\n", descP->sock, reason, saveErrno) ); + + result = esock_make_error(env, reason); + + } else { + SSDBG( descP, + ("SOCKET", "esock_ioctl_sifflags {%d} -> " + "updated flags successfully set\r\n", + descP->sock) ); + result = esock_atom_ok; + } + + /* We know that if esock_decode_string is successful, + * we have "some" form of string, and therefor memory + * has been allocated (and need to be freed)... */ + FREE(ifn); + + } else { + result = enif_make_badarg(env); + } + } + + SSDBG( descP, + ("SOCKET", "esock_ioctl_sifflags {%d} -> done with result: " + "\r\n %T" + "\r\n", + descP->sock, result) ); + + return result; + +} +#endif + + + +/* =========================================================================== + * ioctl utility functions + * + */ + +static +ERL_NIF_TERM encode_ioctl_ifconf(ErlNifEnv* env, + ESockDescriptor* descP, + struct ifconf* ifcP) +{ + ERL_NIF_TERM result; + unsigned int len = ((ifcP == NULL) ? 0 : + (ifcP->ifc_len / sizeof(struct ifreq))); + + SSDBG( descP, + ("SOCKET", "encode_ioctl_ifconf -> entry (when len = %d)\r\n", len) ); + + if (len > 0) { + ERL_NIF_TERM* array = MALLOC(len * sizeof(ERL_NIF_TERM)); + unsigned int i = 0; + struct ifreq* p = ifcP->ifc_req; + + for (i = 0 ; i < len ; i++) { + SSDBG( descP, + ("SOCKET", "encode_ioctl_ifconf -> encode ifreq entry %d\r\n", i) ); + array[i] = encode_ioctl_ifconf_ifreq(env, descP, &p[i]); + } + + SSDBG( descP, + ("SOCKET", "encode_ioctl_ifconf -> all entries encoded\r\n", i) ); + + result = esock_make_ok2(env, MKLA(env, array, len)); + FREE(array); + + } else { + + result = esock_make_ok2(env, MKEL(env)); + + } + + return result; +} + + +#if defined(SIOCGIFMAP) && defined(ESOCK_USE_IFMAP) +static +ERL_NIF_TERM encode_ioctl_ifrmap(ErlNifEnv* env, + ESockDescriptor* descP, + struct ifmap* mapP) +{ + ERL_NIF_TERM mapKeys[] = {atom_mem_start, + atom_mem_end, + atom_base_addr, + atom_irq, + atom_dma, + atom_port}; + ERL_NIF_TERM mapVals[] = {MKUL(env, mapP->mem_start), + MKUL(env, mapP->mem_end), + MKUI(env, mapP->base_addr), + MKUI(env, mapP->irq), + MKUI(env, mapP->dma), + MKUI(env, mapP->port)}; + unsigned int numMapKeys = NUM(mapKeys); + unsigned int numMapVals = NUM(mapVals); + ERL_NIF_TERM emap; + + ESOCK_ASSERT( numMapVals == numMapKeys ); + ESOCK_ASSERT( MKMA(env, mapKeys, mapVals, numMapKeys, &emap) ); + + SSDBG( descP, ("SOCKET", "encode_ioctl_ifrmap -> done with" + "\r\n Map: %T" + "\r\n", emap) ); + + return esock_make_ok2(env, emap);; +} +#endif + + +#if defined(SIOCGIFHWADDR) && defined(ESOCK_USE_HWADDR) +static +ERL_NIF_TERM encode_ioctl_hwaddr(ErlNifEnv* env, + ESockDescriptor* descP, + struct sockaddr* addrP) +{ + ERL_NIF_TERM eaddr; + SOCKLEN_T sz = sizeof(struct sockaddr); + + esock_encode_hwsockaddr(env, addrP, sz, &eaddr); + + SSDBG( descP, ("SOCKET", "encode_ioctl_ifraddr -> done with" + "\r\n Sock Addr: %T" + "\r\n", eaddr) ); + + return esock_make_ok2(env, eaddr);; +} +#endif + + +static +ERL_NIF_TERM encode_ioctl_ifraddr(ErlNifEnv* env, + ESockDescriptor* descP, + struct sockaddr* addrP) +{ + ERL_NIF_TERM eaddr; + unsigned int sz = sizeof(ESockAddress); + + esock_encode_sockaddr(env, (ESockAddress*) addrP, sz, &eaddr); + + SSDBG( descP, ("SOCKET", "encode_ioctl_ifraddr -> done with" + "\r\n Sock Addr: %T" + "\r\n", eaddr) ); + + return esock_make_ok2(env, eaddr);; +} + + +static +ERL_NIF_TERM encode_ioctl_flags(ErlNifEnv* env, + ESockDescriptor* descP, + short flags) +{ + int i, flag, num = NUM(ioctl_flags); + ERL_NIF_TERM eflags, eflag; + SocketTArray ta = TARRAY_CREATE(20); // Just to be on the safe side + + if (flags == 0) { + eflags = MKEL(env); + } else { + for (i = 0; (i < num) && (flags != 0); i++) { + flag = ioctl_flags[i].flag; + if ((flag != 0) && ((flags & flag) == flag)) { + eflag = *(ioctl_flags[i].name); + flags &= ~flag; + + SSDBG( descP, ("SOCKET", "encode_ioctl_flags {%d} -> " + "\r\n i: %d" + "\r\n found flag: %T (%d)" + "\r\n remaining flags: %d" + "\r\n", descP->sock, i, eflag, flag, flags) ); + + TARRAY_ADD(ta, eflag); + } + } + if (flags != 0) { + + SSDBG( descP, ("SOCKET", "encode_ioctl_flags {%d} -> unknown flag(s): %d" + "\r\n", descP->sock, flags) ); + + TARRAY_ADD(ta, MKI(env, flags)); + } + + TARRAY_TOLIST(ta, env, &eflags); + } + + + SSDBG( descP, ("SOCKET", "encode_ioctl_flags -> done with" + "\r\n Flags: %T (%d)" + "\r\n", eflags, flags) ); + + return esock_make_ok2(env, eflags);; +} + + +static +BOOLEAN_T decode_ioctl_sockaddr(ErlNifEnv* env, + ESockDescriptor* descP, + ERL_NIF_TERM eaddr, + ESockAddress* addr) +{ + SOCKLEN_T addrLen; + BOOLEAN_T result; + + result = esock_decode_sockaddr(env, eaddr, (ESockAddress*) addr, &addrLen); + + VOID(addrLen); + + SSDBG( descP, + ("SOCKET", "esock_decode_ioctl_sockaddr {%d} -> decode result: %s" + "\r\n", descP->sock, B2S(result)) ); + + return result; +} + + +static +BOOLEAN_T decode_ioctl_mtu(ErlNifEnv* env, + ESockDescriptor* descP, + ERL_NIF_TERM emtu, + int* mtu) +{ + BOOLEAN_T result; + + if (! GET_INT(env, emtu, mtu)) { + result = FALSE; + } else { + result = TRUE; + } + + SSDBG( descP, + ("SOCKET", "esock_decode_ioctl_mtu {%d} -> decode result: %s" + "\r\n", descP->sock, B2S(result)) ); + + return result; +} + + +#if defined(SIOCSIFTXQLEN) +static +BOOLEAN_T decode_ioctl_txqlen(ErlNifEnv* env, + ESockDescriptor* descP, + ERL_NIF_TERM etxqlen, + int* txqlen) +{ + return decode_ioctl_ivalue(env, descP, etxqlen, txqlen); +} +#endif + +/* All uses of the function should be added. For instance: + * #if defined(SIOCGIFTXQLEN) || defined(FOOBAR) || defined(YXA) + */ +#if defined(SIOCGIFTXQLEN) +static +BOOLEAN_T decode_ioctl_ivalue(ErlNifEnv* env, + ESockDescriptor* descP, + ERL_NIF_TERM eivalue, + int* ivalue) +{ + BOOLEAN_T result; + + if (! GET_INT(env, eivalue, ivalue)) { + result = FALSE; + } else { + result = TRUE; + } + + SSDBG( descP, + ("SOCKET", "esock_decode_ioctl_ivalue {%d} -> decode result: %s" + "\r\n", descP->sock, B2S(result)) ); + + return result; +} +#endif + + +static +BOOLEAN_T decode_ioctl_flags(ErlNifEnv* env, + ESockDescriptor* descP, + ERL_NIF_TERM eflags, + short* flags) +{ + ERL_NIF_TERM key, value; + ErlNifMapIterator iter; + int tmpFlags = (int) *flags; // Current value + int flag; + + SSDBG( descP, + ("SOCKET", "decode_ioctl_flags {%d} -> entry with" + "\r\n flags: %d" + "\r\n", + descP->sock, tmpFlags) ); + + enif_map_iterator_create(env, eflags, &iter, ERL_NIF_MAP_ITERATOR_FIRST); + + while (enif_map_iterator_get_pair(env, &iter, &key, &value)) { + + /* Convert key (eflag) to int */ + if (! GET_INT(env, key, &flag)) { + enif_map_iterator_destroy(env, &iter); + return FALSE; + } + + // Update flag + if (COMPARE(value, esock_atom_true) == 0) { + SSDBG( descP, + ("SOCKET", "decode_ioctl_flags {%d} -> set %d\r\n", + descP->sock, flag) ); + tmpFlags |= flag; + } else { + SSDBG( descP, + ("SOCKET", "decode_ioctl_flags {%d} -> reset %d\r\n", + descP->sock, flag) ); + tmpFlags &= ~flag; + } + + enif_map_iterator_next(env, &iter); + } + + enif_map_iterator_destroy(env, &iter); + + SSDBG( descP, + ("SOCKET", "decode_ioctl_flags {%d} -> done with" + "\r\n (new) flags: %d" + "\r\n", + descP->sock, tmpFlags) ); + + *flags = (short) tmpFlags; + + return TRUE; +} + + +static +ERL_NIF_TERM encode_ioctl_ivalue(ErlNifEnv* env, + ESockDescriptor* descP, + int ivalue) +{ + ERL_NIF_TERM eivalue = MKI(env, ivalue); + + SSDBG( descP, ("SOCKET", "encode_ioctl_ivalue -> done with" + "\r\n iValue: %T (%d)" + "\r\n", eivalue, ivalue) ); + + return esock_make_ok2(env, eivalue);; +} + +static +ERL_NIF_TERM encode_ioctl_ifconf_ifreq(ErlNifEnv* env, + ESockDescriptor* descP, + struct ifreq* ifrP) +{ + ERL_NIF_TERM ename, eaddr; + + ESOCK_ASSERT( ifrP != NULL ); + + SSDBG( descP, ("SOCKET", "encode_ioctl_ifconf_ifreq -> encode name\r\n") ); + ename = encode_ioctl_ifreq_name(env, ifrP->ifr_name); + + SSDBG( descP, ("SOCKET", "encode_ioctl_ifconf_ifreq -> encode sockaddr\r\n") ); + eaddr = encode_ioctl_ifreq_sockaddr(env, &ifrP->ifr_addr); + + SSDBG( descP, ("SOCKET", "encode_ioctl_ifconf_ifreq -> make ifreq map with" + "\r\n Name: %T" + "\r\n Sock Addr: %T" + "\r\n", ename, eaddr) ); + return make_ifreq(env, ename, esock_atom_addr, eaddr); +} + +static +ERL_NIF_TERM encode_ioctl_ifreq_name(ErlNifEnv* env, + char* name) +{ + return ((name == NULL) ? esock_atom_undefined : MKS(env, name)); +} + +static +ERL_NIF_TERM encode_ioctl_ifreq_sockaddr(ErlNifEnv* env, struct sockaddr* sa) +{ + ERL_NIF_TERM esa; + + if (sa != NULL) { + unsigned int sz = sizeof(ESockAddress); + + esock_encode_sockaddr(env, (ESockAddress*) sa, sz, &esa); + + } else { + esa = esock_atom_undefined; + } + + return esa; +} + +/* The ifreq structure *always* contain a name + * and *one* other element. The second element + * depend on the ioctl request. + */ +static +ERL_NIF_TERM make_ifreq(ErlNifEnv* env, + ERL_NIF_TERM name, + ERL_NIF_TERM key2, + ERL_NIF_TERM val2) +{ + ERL_NIF_TERM keys[2]; + ERL_NIF_TERM vals[2]; + ERL_NIF_TERM res; + + keys[0] = esock_atom_name; + vals[0] = name; + + keys[1] = key2; + vals[1] = val2; + + ESOCK_ASSERT( MKMA(env, keys, vals, NUM(keys), &res) ); + + return res; +} + #endif // #ifndef __WIN32__ @@ -13241,7 +15178,7 @@ ERL_NIF_TERM send_check_fail(ErlNifEnv* env, int saveErrno, ERL_NIF_TERM sockRef) { - ERL_NIF_TERM reason; + ERL_NIF_TERM reason; ESOCK_CNT_INC(env, descP, sockRef, atom_write_fails, &descP->writeFails, 1); @@ -15797,7 +17734,7 @@ void encode_msg_flags(ErlNifEnv* env, if (msgFlags == 0) { *flags = MKEL(env); } else { - size_t n; + size_t n; SocketTArray ta = TARRAY_CREATE(10); // Just to be on the safe side for (n = 0; n < NUM(msg_flags); n++) { @@ -17093,7 +19030,7 @@ REQ_SEARCH4PID_FUNCS != NULL ); \ reqP = &e->data; \ reqP->pid = pid; \ - ESOCK_ASSERT( MONP("reader_push -> " #F " request", \ + ESOCK_ASSERT( MONP(#F "_push -> " #F " request", \ env, descP, &pid, &reqP->mon) == 0 ); \ reqP->env = esock_alloc_env(#F "_push"); \ reqP->ref = CP_TERM(reqP->env, ref); \ @@ -18232,6 +20169,9 @@ ErlNifFunc esock_funcs[] = {"nif_getopt", 4, nif_getopt, 0}, {"nif_sockname", 1, nif_sockname, 0}, {"nif_peername", 1, nif_peername, 0}, + {"nif_ioctl", 2, nif_ioctl, ERL_NIF_DIRTY_JOB_IO_BOUND}, + {"nif_ioctl", 3, nif_ioctl, ERL_NIF_DIRTY_JOB_IO_BOUND}, + {"nif_ioctl", 4, nif_ioctl, ERL_NIF_DIRTY_JOB_IO_BOUND}, /* Misc utility functions */ diff --git a/erts/emulator/nifs/common/socket_int.h b/erts/emulator/nifs/common/socket_int.h index b892eeefb6..cc8584ee7e 100644 --- a/erts/emulator/nifs/common/socket_int.h +++ b/erts/emulator/nifs/common/socket_int.h @@ -181,7 +181,16 @@ typedef int BOOLEAN_T; GLOBAL_ATOM_DEF(addrform); \ GLOBAL_ATOM_DEF(add_membership); \ GLOBAL_ATOM_DEF(add_source_membership); \ + GLOBAL_ATOM_DEF(allmulti); \ GLOBAL_ATOM_DEF(any); \ + GLOBAL_ATOM_DEF(arphrd_dlci); \ + GLOBAL_ATOM_DEF(arphrd_ether); \ + GLOBAL_ATOM_DEF(arphrd_frelay); \ + GLOBAL_ATOM_DEF(arphrd_ieee802); \ + GLOBAL_ATOM_DEF(arphrd_ieee1394); \ + GLOBAL_ATOM_DEF(arphrd_loopback); \ + GLOBAL_ATOM_DEF(arphrd_netrom); \ + GLOBAL_ATOM_DEF(arphrd_none); \ GLOBAL_ATOM_DEF(associnfo); \ GLOBAL_ATOM_DEF(authhdr); \ GLOBAL_ATOM_DEF(auth_active_key); \ @@ -191,11 +200,14 @@ typedef int BOOLEAN_T; GLOBAL_ATOM_DEF(auth_key); \ GLOBAL_ATOM_DEF(auth_level); \ GLOBAL_ATOM_DEF(autoclose); \ + GLOBAL_ATOM_DEF(automedia); \ GLOBAL_ATOM_DEF(bad_data); \ GLOBAL_ATOM_DEF(bindtodevice); \ GLOBAL_ATOM_DEF(block_source); \ GLOBAL_ATOM_DEF(broadcast); \ GLOBAL_ATOM_DEF(busy_poll); \ + GLOBAL_ATOM_DEF(cantconfig); \ + GLOBAL_ATOM_DEF(chaos); \ GLOBAL_ATOM_DEF(checksum); \ GLOBAL_ATOM_DEF(close); \ GLOBAL_ATOM_DEF(cmsg_cloexec); \ @@ -218,9 +230,13 @@ typedef int BOOLEAN_T; GLOBAL_ATOM_DEF(domain); \ GLOBAL_ATOM_DEF(dontfrag); \ GLOBAL_ATOM_DEF(dontroute); \ + GLOBAL_ATOM_DEF(dormant); \ GLOBAL_ATOM_DEF(drop_membership); \ GLOBAL_ATOM_DEF(drop_source_membership); \ GLOBAL_ATOM_DEF(dstopts); \ + GLOBAL_ATOM_DEF(dying); \ + GLOBAL_ATOM_DEF(dynamic); \ + GLOBAL_ATOM_DEF(echo); \ GLOBAL_ATOM_DEF(egp); \ GLOBAL_ATOM_DEF(enotsup); \ GLOBAL_ATOM_DEF(eor); \ @@ -249,6 +265,7 @@ typedef int BOOLEAN_T; GLOBAL_ATOM_DEF(icmp6); \ GLOBAL_ATOM_DEF(ifindex); \ GLOBAL_ATOM_DEF(igmp); \ + GLOBAL_ATOM_DEF(implink); \ GLOBAL_ATOM_DEF(inet); \ GLOBAL_ATOM_DEF(inet6); \ GLOBAL_ATOM_DEF(info); \ @@ -267,19 +284,27 @@ typedef int BOOLEAN_T; GLOBAL_ATOM_DEF(keepidle); \ GLOBAL_ATOM_DEF(keepintvl); \ GLOBAL_ATOM_DEF(kernel); \ + GLOBAL_ATOM_DEF(knowsepoch); \ GLOBAL_ATOM_DEF(leave_group); \ GLOBAL_ATOM_DEF(level); \ GLOBAL_ATOM_DEF(linger); \ + GLOBAL_ATOM_DEF(link); \ + GLOBAL_ATOM_DEF(link0); \ + GLOBAL_ATOM_DEF(link1); \ + GLOBAL_ATOM_DEF(link2); \ GLOBAL_ATOM_DEF(local); \ GLOBAL_ATOM_DEF(local_auth_chunks); \ - GLOBAL_ATOM_DEF(loopback); \ + GLOBAL_ATOM_DEF(loopback); \ GLOBAL_ATOM_DEF(lowdelay); \ + GLOBAL_ATOM_DEF(lower_up); \ GLOBAL_ATOM_DEF(mark); \ + GLOBAL_ATOM_DEF(master); \ GLOBAL_ATOM_DEF(maxburst); \ GLOBAL_ATOM_DEF(maxseg); \ GLOBAL_ATOM_DEF(md5sig); \ GLOBAL_ATOM_DEF(mincost); \ GLOBAL_ATOM_DEF(minttl); \ + GLOBAL_ATOM_DEF(monitor); \ GLOBAL_ATOM_DEF(more); \ GLOBAL_ATOM_DEF(msfilter); \ GLOBAL_ATOM_DEF(mtu); \ @@ -290,13 +315,19 @@ typedef int BOOLEAN_T; GLOBAL_ATOM_DEF(multicast_if); \ GLOBAL_ATOM_DEF(multicast_loop); \ GLOBAL_ATOM_DEF(multicast_ttl); \ + GLOBAL_ATOM_DEF(name); \ + GLOBAL_ATOM_DEF(noarp); \ GLOBAL_ATOM_DEF(nodelay); \ GLOBAL_ATOM_DEF(nodefrag); \ + GLOBAL_ATOM_DEF(nogroup); \ + GLOBAL_ATOM_DEF(none); \ GLOBAL_ATOM_DEF(noopt); \ GLOBAL_ATOM_DEF(nopush); \ GLOBAL_ATOM_DEF(nosignal); \ + GLOBAL_ATOM_DEF(notrailers); \ GLOBAL_ATOM_DEF(not_found); \ GLOBAL_ATOM_DEF(not_owner); \ + GLOBAL_ATOM_DEF(oactive); \ GLOBAL_ATOM_DEF(ok); \ GLOBAL_ATOM_DEF(oob); \ GLOBAL_ATOM_DEF(oobinline); \ @@ -316,11 +347,16 @@ typedef int BOOLEAN_T; GLOBAL_ATOM_DEF(pktinfo); \ GLOBAL_ATOM_DEF(pktoptions); \ GLOBAL_ATOM_DEF(pkttype); \ + GLOBAL_ATOM_DEF(ppromisc); \ + GLOBAL_ATOM_DEF(pointopoint); \ GLOBAL_ATOM_DEF(port); \ GLOBAL_ATOM_DEF(portrange); \ + GLOBAL_ATOM_DEF(portsel); \ GLOBAL_ATOM_DEF(primary_addr); \ GLOBAL_ATOM_DEF(priority); \ + GLOBAL_ATOM_DEF(promisc); \ GLOBAL_ATOM_DEF(protocol); \ + GLOBAL_ATOM_DEF(pup); \ GLOBAL_ATOM_DEF(raw); \ GLOBAL_ATOM_DEF(rcvbuf); \ GLOBAL_ATOM_DEF(rcvbufforce); \ @@ -341,6 +377,7 @@ typedef int BOOLEAN_T; GLOBAL_ATOM_DEF(recvtos); \ GLOBAL_ATOM_DEF(recvttl); \ GLOBAL_ATOM_DEF(reliability); \ + GLOBAL_ATOM_DEF(renaming); \ GLOBAL_ATOM_DEF(reset_streams); \ GLOBAL_ATOM_DEF(retopts); \ GLOBAL_ATOM_DEF(reuseaddr); \ @@ -349,6 +386,7 @@ typedef int BOOLEAN_T; GLOBAL_ATOM_DEF(router_alert); \ GLOBAL_ATOM_DEF(rthdr); \ GLOBAL_ATOM_DEF(rtoinfo); \ + GLOBAL_ATOM_DEF(running); \ GLOBAL_ATOM_DEF(rxq_ovfl); \ GLOBAL_ATOM_DEF(scope_id); \ GLOBAL_ATOM_DEF(sctp); \ @@ -362,6 +400,8 @@ typedef int BOOLEAN_T; GLOBAL_ATOM_DEF(seqpacket); \ GLOBAL_ATOM_DEF(setfib); \ GLOBAL_ATOM_DEF(set_peer_primary_addr); \ + GLOBAL_ATOM_DEF(simplex); \ + GLOBAL_ATOM_DEF(slave); \ GLOBAL_ATOM_DEF(sndbuf); \ GLOBAL_ATOM_DEF(sndbufforce); \ GLOBAL_ATOM_DEF(sndlowat); \ @@ -370,6 +410,7 @@ typedef int BOOLEAN_T; GLOBAL_ATOM_DEF(socket_tag); \ GLOBAL_ATOM_DEF(spec_dst); \ GLOBAL_ATOM_DEF(status); \ + GLOBAL_ATOM_DEF(staticarp); \ GLOBAL_ATOM_DEF(stream); \ GLOBAL_ATOM_DEF(syncnt); \ GLOBAL_ATOM_DEF(tclass); \ @@ -387,6 +428,7 @@ typedef int BOOLEAN_T; GLOBAL_ATOM_DEF(undefined); \ GLOBAL_ATOM_DEF(unicast_hops); \ GLOBAL_ATOM_DEF(unspec); \ + GLOBAL_ATOM_DEF(up); \ GLOBAL_ATOM_DEF(usec); \ GLOBAL_ATOM_DEF(user); \ GLOBAL_ATOM_DEF(user_timeout); \ diff --git a/erts/emulator/nifs/common/socket_util.c b/erts/emulator/nifs/common/socket_util.c index 66db2cf27e..94a5e3e3dc 100644 --- a/erts/emulator/nifs/common/socket_util.c +++ b/erts/emulator/nifs/common/socket_util.c @@ -33,6 +33,9 @@ #include <ctype.h> #include <time.h> #include <stddef.h> +#include <sys/types.h> +#include <sys/socket.h> +#include <net/if_arp.h> #include "socket_int.h" #include "sys.h" @@ -371,56 +374,165 @@ void esock_encode_sockaddr(ErlNifEnv* env, SOCKLEN_T addrLen, ERL_NIF_TERM* eSockAddr) { - int family; + int family; - // Sanity check - if (addrLen < (char *)&sockAddrP->sa.sa_data - (char *)sockAddrP) { - // We got crap, cannot even know the address family - esock_encode_sockaddr_broken(env, &sockAddrP->sa, addrLen, eSockAddr); - return; - } - family = sockAddrP->ss.ss_family; + // Sanity check + if (addrLen < (char *)&sockAddrP->sa.sa_data - (char *)sockAddrP) { + // We got crap, cannot even know the address family + esock_encode_sockaddr_broken(env, &sockAddrP->sa, addrLen, eSockAddr); + return; + } + family = sockAddrP->ss.ss_family; - UDBG( ("SUTIL", "esock_encode_sockaddr -> entry with" - "\r\n family: %d" - "\r\n addrLen: %d" - "\r\n", family, addrLen) ); + UDBG( ("SUTIL", "esock_encode_sockaddr -> entry with" + "\r\n family: %d" + "\r\n addrLen: %d" + "\r\n", family, addrLen) ); - switch (family) { - case AF_INET: - esock_encode_sockaddr_in(env, &sockAddrP->in4, addrLen, eSockAddr); - break; + switch (family) { + case AF_INET: + esock_encode_sockaddr_in(env, &sockAddrP->in4, addrLen, eSockAddr); + break; #if defined(HAVE_IN6) && defined(AF_INET6) - case AF_INET6: - esock_encode_sockaddr_in6(env, &sockAddrP->in6, addrLen, eSockAddr); - break; + case AF_INET6: + esock_encode_sockaddr_in6(env, &sockAddrP->in6, addrLen, eSockAddr); + break; #endif #ifdef HAS_AF_LOCAL - case AF_LOCAL: - esock_encode_sockaddr_un(env, &sockAddrP->un, addrLen, eSockAddr); - break; + case AF_LOCAL: + esock_encode_sockaddr_un(env, &sockAddrP->un, addrLen, eSockAddr); + break; #endif #ifdef AF_UNSPEC - case AF_UNSPEC: - esock_encode_sockaddr_native(env, &sockAddrP->sa, - addrLen, esock_atom_unspec, eSockAddr); - break; + case AF_UNSPEC: + esock_encode_sockaddr_native(env, &sockAddrP->sa, + addrLen, esock_atom_unspec, eSockAddr); + break; #endif #if defined(HAVE_NETPACKET_PACKET_H) - case AF_PACKET: - esock_encode_sockaddr_ll(env, &sockAddrP->ll, addrLen, eSockAddr); - break; + case AF_PACKET: + esock_encode_sockaddr_ll(env, &sockAddrP->ll, addrLen, eSockAddr); + break; #endif - default: - esock_encode_sockaddr_native(env, &sockAddrP->sa, - addrLen, MKI(env, family), eSockAddr); - break; - } +#if defined(AF_IMPLINK) + case AF_IMPLINK: + esock_encode_sockaddr_native(env, &sockAddrP->sa, + addrLen, esock_atom_implink, eSockAddr); + break; +#endif + +#if defined(AF_PUP) + case AF_PUP: + esock_encode_sockaddr_native(env, &sockAddrP->sa, + addrLen, esock_atom_pup, eSockAddr); + break; +#endif + +#if defined(AF_CHAOS) + case AF_CHAOS: + esock_encode_sockaddr_native(env, &sockAddrP->sa, + addrLen, esock_atom_chaos, eSockAddr); + break; +#endif + +#if defined(AF_LINK) + case AF_LINK: + esock_encode_sockaddr_native(env, &sockAddrP->sa, + addrLen, esock_atom_link, eSockAddr); + break; +#endif + + default: + esock_encode_sockaddr_native(env, &sockAddrP->sa, + addrLen, MKI(env, family), eSockAddr); + break; + } +} + + + +extern +void esock_encode_hwsockaddr(ErlNifEnv* env, + struct sockaddr* sockAddrP, + SOCKLEN_T addrLen, + ERL_NIF_TERM* eSockAddr) +{ + ERL_NIF_TERM efamily; + int family; + + // Sanity check + if (addrLen < (char *)&sockAddrP->sa_data - (char *)sockAddrP) { + // We got crap, cannot even know the address family + esock_encode_sockaddr_broken(env, sockAddrP, addrLen, eSockAddr); + return; + } + family = sockAddrP->sa_family; + + UDBG( ("SUTIL", "esock_encode_hwsockaddr -> entry with" + "\r\n family: %d" + "\r\n addrLen: %d" + "\r\n", family, addrLen) ); + + switch (family) { +#if defined(ARPHRD_NETROM) + case ARPHRD_NETROM: + efamily = esock_atom_arphrd_netrom; + break; +#endif + +#if defined(ARPHRD_ETHER) + case ARPHRD_ETHER: + efamily = esock_atom_arphrd_ether; + break; +#endif + +#if defined(ARPHRD_IEEE802) + case ARPHRD_IEEE802: + efamily = esock_atom_arphrd_ieee802; + break; +#endif + +#if defined(ARPHRD_DLCI) + case ARPHRD_DLCI: + efamily = esock_atom_arphrd_dlci; + break; +#endif + +#if defined(ARPHRD_FRELAY) + case ARPHRD_FRELAY: + efamily = esock_atom_arphrd_frelay; + break; +#endif + +#if defined(ARPHRD_IEEE1394) + case ARPHRD_IEEE1394: + efamily = esock_atom_arphrd_ieee1394; + break; +#endif + +#if defined(ARPHRD_LOOPBACK) + case ARPHRD_LOOPBACK: + efamily = esock_atom_arphrd_loopback; + break; +#endif + +#if defined(ARPHRD_NONE) + case ARPHRD_NONE: + efamily = esock_atom_arphrd_none; + break; +#endif + + default: + efamily = MKI(env, family); + break; + } + + esock_encode_sockaddr_native(env, sockAddrP, addrLen, efamily, eSockAddr); } @@ -2190,10 +2302,12 @@ void make_sockaddr_ll(ErlNifEnv* env, extern -ERL_NIF_TERM esock_make_new_binary(ErlNifEnv *env, void *buf, size_t size) { +ERL_NIF_TERM esock_make_new_binary(ErlNifEnv *env, void *buf, size_t size) +{ ERL_NIF_TERM term; sys_memcpy(enif_make_new_binary(env, size, &term), buf, size); + return term; } @@ -2212,7 +2326,8 @@ ERL_NIF_TERM esock_make_new_binary(ErlNifEnv *env, void *buf, size_t size) { * badarg */ extern -BOOLEAN_T esock_is_integer(ErlNifEnv *env, ERL_NIF_TERM term) { +BOOLEAN_T esock_is_integer(ErlNifEnv *env, ERL_NIF_TERM term) +{ double d; /* Test that it is a number() but not a float(), diff --git a/erts/emulator/nifs/common/socket_util.h b/erts/emulator/nifs/common/socket_util.h index 61b237383c..af58c15377 100644 --- a/erts/emulator/nifs/common/socket_util.h +++ b/erts/emulator/nifs/common/socket_util.h @@ -36,6 +36,7 @@ #define LONG(L) ((long) (L)) #define ULONG(L) ((unsigned long) (L)) #define SZT(I) ((size_t) (I)) +#define VOID(D) ((void) (D)) #define VOIDP(P) ((void*) (P)) #define CHARP(P) ((char*) (P)) #define UCHARP(P) ((unsigned char*) (P)) @@ -74,7 +75,11 @@ void esock_encode_sockaddr(ErlNifEnv* env, ESockAddress* sockAddrP, SOCKLEN_T addrLen, ERL_NIF_TERM* eSockAddr); - +extern +void esock_encode_hwsockaddr(ErlNifEnv* env, + struct sockaddr* sockAddrP, + SOCKLEN_T addrLen, + ERL_NIF_TERM* eSockAddr); extern BOOLEAN_T esock_decode_sockaddr_in(ErlNifEnv* env, ERL_NIF_TERM eSockAddr, diff --git a/erts/emulator/sys/common/erl_mseg.c b/erts/emulator/sys/common/erl_mseg.c index 030e5b00a7..7e0200bf34 100644 --- a/erts/emulator/sys/common/erl_mseg.c +++ b/erts/emulator/sys/common/erl_mseg.c @@ -193,7 +193,7 @@ static ErtsAlgndMsegAllctr_t *aligned_mseg_allctr; (&aligned_mseg_allctr[(IX)].mseg_alloc) #define ERTS_MSEG_ALLCTR_SS() \ - ERTS_MSEG_ALLCTR_IX((int) erts_get_scheduler_id()) + ERTS_MSEG_ALLCTR_IX((int) erts_get_thr_alloc_ix()) #define ERTS_MSEG_ALLCTR_OPT(OPT) \ ((OPT)->sched_spec ? ERTS_MSEG_ALLCTR_SS() : ERTS_MSEG_ALLCTR_IX(0)) @@ -1391,8 +1391,10 @@ erts_mseg_init(ErtsMsegInit_t *init) int i; UWord x; - no_mseg_allocators = init->nos + 1; - + no_mseg_allocators = 1; /* Global instance */ + no_mseg_allocators += init->nos; /* Scheduler specific instances */ + no_mseg_allocators += init->ndai; /* Dirty alloc instances */ + x = (UWord) malloc(sizeof(ErtsAlgndMsegAllctr_t) *no_mseg_allocators + (ERTS_CACHE_LINE_SIZE-1)); @@ -1423,7 +1425,7 @@ erts_mseg_init(ErtsMsegInit_t *init) ma->is_init_done = 0; - if (i != 0) + if (i != 0 && i <= init->nos) ma->is_thread_safe = 0; else { ma->is_thread_safe = 1; diff --git a/erts/emulator/sys/common/erl_mseg.h b/erts/emulator/sys/common/erl_mseg.h index ea9060ddac..8d5e627664 100644 --- a/erts/emulator/sys/common/erl_mseg.h +++ b/erts/emulator/sys/common/erl_mseg.h @@ -59,6 +59,7 @@ typedef struct { Uint rmcbf; Uint mcs; Uint nos; + Uint ndai; ErtsMMapInit dflt_mmap; ErtsMMapInit literal_mmap; } ErtsMsegInit_t; @@ -68,7 +69,8 @@ typedef struct { 4*1024*1024, /* amcbf: Absolute max cache bad fit */ \ 20, /* rmcbf: Relative max cache bad fit */ \ 10, /* mcs: Max cache size */ \ - 1000, /* cci: Cache check interval */ \ + 0, /* nos: Number of schedulers */ \ + 0, /* ndai: Number of dirty alloc instances*/ \ ERTS_MMAP_INIT_DEFAULT_INITER, \ ERTS_MMAP_INIT_LITERAL_INITER, \ } diff --git a/erts/emulator/sys/common/erl_poll.c b/erts/emulator/sys/common/erl_poll.c index b692f0e4e3..b7f7d295c4 100644 --- a/erts/emulator/sys/common/erl_poll.c +++ b/erts/emulator/sys/common/erl_poll.c @@ -71,6 +71,12 @@ # define _DARWIN_UNLIMITED_SELECT #endif +/* + * According to posix, select() implementations should + * support a max timeout value of at least 31 days. + */ +#define ERTS_SELECT_MAX_TV_SEC__ (31*24*60*60-1) + #ifndef WANT_NONBLOCKING # define WANT_NONBLOCKING #endif @@ -1757,6 +1763,17 @@ get_timeout_timeval(ErtsPollSet *ps, } else { ErtsMonotonicTime sec = timeout/(1000*1000); + if (sec >= ERTS_SELECT_MAX_TV_SEC__) { + tvp->tv_sec = ERTS_SELECT_MAX_TV_SEC__; + tvp->tv_usec = 0; + /* + * If we actually should time out on + * this (huge) timeout, the select() call + * will be restarted with a newly calculated + * timeout after verifying the timeout... + */ + return 1; + } tvp->tv_sec = sec; tvp->tv_usec = timeout - sec*(1000*1000); diff --git a/erts/emulator/test/Makefile b/erts/emulator/test/Makefile index 34a73a6f1d..80b0058663 100644 --- a/erts/emulator/test/Makefile +++ b/erts/emulator/test/Makefile @@ -161,6 +161,9 @@ ERL_FILES= $(MODULES:%=%.erl) TARGET_FILES = $(MODULES:%=$(EBIN)/%.$(EMULATOR)) +KERNEL_MODULES=gen_tcp_dist +KERNEL_ERL_FILES=$(ERL_TOP)/lib/kernel/examples/gen_tcp_dist/src/gen_tcp_dist.erl + EMAKEFILE=Emakefile TEST_SPEC_FILES= emulator.spec \ @@ -184,9 +187,9 @@ ERL_COMPILE_FLAGS += # Targets # ---------------------------------------------------- -make_emakefile: $(NO_OPT_ERL_FILES) $(NATIVE_ERL_FILES) +make_emakefile: $(NO_OPT_ERL_FILES) $(NATIVE_ERL_FILES) $(KERNEL_ERL_FILES) $(ERL_TOP)/make/make_emakefile $(ERL_COMPILE_FLAGS) +compressed -o$(EBIN) \ - $(MODULES) >> $(EMAKEFILE) + $(MODULES) $(KERNEL_MODULES) >> $(EMAKEFILE) $(ERL_TOP)/make/make_emakefile +no_copt +no_postopt +no_ssa_opt +no_bsm_opt \ $(ERL_COMPILE_FLAGS) -o$(EBIN) $(NO_OPT_MODULES) >> $(EMAKEFILE) $(ERL_TOP)/make/make_emakefile $(ERL_COMPILE_FLAGS) \ @@ -227,6 +230,7 @@ release_tests_spec: make_emakefile $(ERL_FILES) "$(RELSYSDIR)" $(INSTALL_DATA) $(NO_OPT_ERL_FILES) "$(RELSYSDIR)" $(INSTALL_DATA) $(NATIVE_ERL_FILES) "$(RELSYSDIR)" + $(INSTALL_DATA) $(KERNEL_ERL_FILES) "$(RELSYSDIR)" chmod -R u+w "$(RELSYSDIR)" tar cf - *_SUITE_data property_test | (cd "$(RELSYSDIR)"; tar xf -) diff --git a/erts/emulator/test/binary_SUITE.erl b/erts/emulator/test/binary_SUITE.erl index 2840eb3dd0..f4a61209d3 100644 --- a/erts/emulator/test/binary_SUITE.erl +++ b/erts/emulator/test/binary_SUITE.erl @@ -1049,8 +1049,14 @@ total_memory() -> end. run_when_enough_resources(Fun) -> + DemandGb = case erlang:system_info(build_type) of + Gb when Gb =:= valgrind; Gb =:= asan -> + 30; + _ -> + 15 + end, case {total_memory(), erlang:system_info(wordsize)} of - {Mem, 8} when is_integer(Mem) andalso Mem >= 15 -> + {Mem, 8} when is_integer(Mem) andalso Mem >= DemandGb -> Fun(); {Mem, WordSize} -> {skipped, diff --git a/erts/emulator/test/bs_construct_SUITE.erl b/erts/emulator/test/bs_construct_SUITE.erl index 81107c5644..a30c394971 100644 --- a/erts/emulator/test/bs_construct_SUITE.erl +++ b/erts/emulator/test/bs_construct_SUITE.erl @@ -132,6 +132,12 @@ l(I_13, I_big1) -> ?T(<<32978297842987249827298387697777669766334937:128/native-integer>>, native_bignum()), + ?T(<<$э/native-utf16,$т/native-utf16,$о/native-utf16," спутник"/native-utf16>>, + native_utf16()), + + ?T(<<$в/native-utf32,"ода"/native-utf32>>, + native_utf32()), + %% Unit tests. ?T(<<<<5:3>>/bitstring>>, <<5:3>>), ?T(<<42,<<7:4>>/binary-unit:4>>, <<42,7:4>>), @@ -152,6 +158,18 @@ native_bignum() -> <<1,0>> -> [217,73,11,176,128,109,231,39,101,170,213,1,177,18,205,129] end. +native_utf16() -> + case <<1:16/native>> of + <<0,1>> -> [4,77,4,66,4,62,0,32,4,65,4,63,4,67,4,66,4,61,4,56,4,58]; + <<1,0>> -> [77,4,66,4,62,4,32,0,65,4,63,4,67,4,66,4,61,4,56,4,58,4] + end. + +native_utf32() -> + case <<1:16/native>> of + <<0,1>> -> [0,0,4,50,0,0,4,62,0,0,4,52,0,0,4,48]; + <<1,0>> -> [50,4,0,0,62,4,0,0,52,4,0,0,48,4,0,0] + end. + evaluate(Str, Vars) -> {ok,Tokens,_} = erl_scan:string(Str ++ " . "), @@ -173,13 +191,13 @@ eval_list([{C_bin, Str, Bytes} | Rest], Vars) -> end. one_test({C_bin, E_bin, Str, Bytes}) when is_list(Bytes) -> - io:format(" ~s, ~p~n", [Str, Bytes]), + io:format(" ~ts, ~p~n", [Str, Bytes]), Bin = list_to_binary(Bytes), if C_bin == Bin -> ok; true -> - io:format("ERROR: Compiled: ~p. Expected ~p. Got ~p.~n", + io:format("ERROR: Compiled: ~p.~nExpected ~p.~nGot ~p.~n", [Str, Bytes, binary_to_list(C_bin)]), ct:fail(comp) end, @@ -187,12 +205,12 @@ one_test({C_bin, E_bin, Str, Bytes}) when is_list(Bytes) -> E_bin == Bin -> ok; true -> - io:format("ERROR: Interpreted: ~p. Expected ~p. Got ~p.~n", + io:format("ERROR: Interpreted: ~p.~nExpected ~p.~nGot ~p.~n", [Str, Bytes, binary_to_list(E_bin)]), ct:fail(comp) end; one_test({C_bin, E_bin, Str, Result}) -> - io:format(" ~s ~p~n", [Str, C_bin]), + io:format(" ~ts ~p~n", [Str, C_bin]), if C_bin == E_bin -> ok; diff --git a/erts/emulator/test/bs_match_misc_SUITE.erl b/erts/emulator/test/bs_match_misc_SUITE.erl index 42992d2f46..bd10284c49 100644 --- a/erts/emulator/test/bs_match_misc_SUITE.erl +++ b/erts/emulator/test/bs_match_misc_SUITE.erl @@ -291,11 +291,15 @@ native(Config) when is_list(Config) -> native_big() -> <<37.33:64/native-float>> = <<37.33:64/big-float>>, <<3974:16/native-integer>> = <<3974:16/big-integer>>, + <<$W/native-utf16>> = id(<<0,$W>>), + <<$W/native-utf32>> = id(<<0,0,0,$W>>), {comment,"Big endian"}. native_little() -> <<37869.32343:64/native-float>> = <<37869.32343:64/little-float>>, <<7974:16/native-integer>> = <<7974:16/little-integer>>, + <<$W/native-utf16>> = id(<<$W,0>>), + <<$W/native-utf32>> = id(<<$W,0,0,0>>), {comment,"Little endian"}. happi(Config) when is_list(Config) -> diff --git a/erts/emulator/test/distribution_SUITE.erl b/erts/emulator/test/distribution_SUITE.erl index 496e80b9ac..8f3c1f3903 100644 --- a/erts/emulator/test/distribution_SUITE.erl +++ b/erts/emulator/test/distribution_SUITE.erl @@ -2558,17 +2558,23 @@ no_epmd(Config) when is_list(Config) -> epmd_module(Config) when is_list(Config) -> %% We need a relay node to test this, since the test node uses the %% standard epmd module. - Sock1 = start_relay_node(epmd_module_node1, "-epmd_module " ++ ?MODULE_STRING), - Node1 = inet_rpc_nodename(Sock1), + {N1,_,S1} = Sock1 = start_relay_node( + epmd_module_node1, + "-epmd_module " ++ ?MODULE_STRING, + [{"ERL_AFLAGS","-sname epmd_module_node1@dummy " ++ os:getenv("ERL_AFLAGS","")}]), + Node1 = inet_rpc_nodename({N1,"dummy",S1}), %% Ask what port it's listening on - it won't have registered with %% epmd. {ok, {ok, Port1}} = do_inet_rpc(Sock1, application, get_env, [kernel, dist_listen_port]), %% Start a second node, passing the port number as a secret %% argument. - Sock2 = start_relay_node(epmd_module_node2, "-epmd_module " ++ ?MODULE_STRING - ++ " -other_node_port " ++ integer_to_list(Port1)), - Node2 = inet_rpc_nodename(Sock2), + {N2,_,S2} = Sock2 = start_relay_node( + epmd_module_node2, + "-epmd_module " ++ ?MODULE_STRING + ++ " -other_node_port " ++ integer_to_list(Port1), + [{"ERL_AFLAGS","-sname epmd_module_node2@dummy " ++ os:getenv("ERL_AFLAGS","")}]), + Node2 = inet_rpc_nodename({N2,"dummy",S2}), %% Node 1 can't ping node 2 {ok, pang} = do_inet_rpc(Sock1, net_adm, ping, [Node2]), {ok, []} = do_inet_rpc(Sock1, erlang, nodes, []), @@ -2608,10 +2614,11 @@ port_please(_Name, _Ip) -> {port, Port, Version} end. -address_please(_Name, _Address, _AddressFamily) -> +address_please(_Name, "dummy", inet) -> %% Use localhost. - IP = {127,0,0,1}, - {ok, IP}. + {ok, {127,0,0,1}}; +address_please(_Name, "dummy", inet6) -> + {ok, {0,0,0,0,0,0,0,1}}. hopefull_data_encoding(Config) when is_list(Config) -> test_hopefull_data_encoding(Config, true), @@ -2965,6 +2972,8 @@ inet_rpc_server_loop(Sock) -> start_relay_node(Node, Args) -> + start_relay_node(Node, Args, []). +start_relay_node(Node, Args, Env) -> Pa = filename:dirname(code:which(?MODULE)), Cookie = "NOT"++atom_to_list(erlang:get_cookie()), {ok, LSock} = gen_tcp:listen(0, [binary, {packet, 4}, {active, false}]), @@ -2973,7 +2982,8 @@ start_relay_node(Node, Args) -> RunArg = "-run " ++ atom_to_list(?MODULE) ++ " inet_rpc_server " ++ Host ++ " " ++ integer_to_list(Port), {ok, NN} = test_server:start_node(Node, peer, - [{args, Args ++ + [{env,Env}, + {args, Args ++ " -setcookie "++Cookie++" -pa "++Pa++" "++ RunArg}]), [N,H] = string:lexemes(atom_to_list(NN),"@"), diff --git a/erts/emulator/test/erts_debug_SUITE.erl b/erts/emulator/test/erts_debug_SUITE.erl index 2c1e83dbdf..2e0373e34f 100644 --- a/erts/emulator/test/erts_debug_SUITE.erl +++ b/erts/emulator/test/erts_debug_SUITE.erl @@ -25,16 +25,19 @@ -export([all/0, suite/0, groups/0, test_size/1,flat_size_big/1,df/1,term_type/1, instructions/1, stack_check/1, alloc_blocks_size/1, + t_copy_shared/1, interpreter_size_bench/1]). -export([do_alloc_blocks_size/0]). + suite() -> [{ct_hooks,[ts_install_cth]}, {timetrap, {minutes, 2}}]. all() -> [test_size, flat_size_big, df, instructions, term_type, + t_copy_shared, stack_check, alloc_blocks_size]. groups() -> @@ -291,3 +294,131 @@ mk_ext_port({NodeName, Creation}, Number) -> mk_ext_ref({NodeName, Creation}, Numbers) -> erts_test_utils:mk_ext_ref({NodeName, Creation}, Numbers). + + +t_copy_shared(_Config) -> + rand:seed(default), + io:format("*** SEED: ~p ***\n", [rand:export_seed()]), + + [copy_shared_term_1(N div 10, CL) || N <- lists:seq(1,100), + CL <- [false, true]], + ok. + +copy_shared_term_1(Size, CopyLit) -> + Term = rand_term(Size), + + %% Note: Printing Term may suppress test failure + %% as it sends a copy to io-server. + %%io:format("rand_term(~p): ~p\n", [Size, printable(Term)]), + + Binary = term_to_binary(Term), + Copy = erts_debug:copy_shared(Term, CopyLit), + test_eq(Term, Copy), + copy_shared_term_2(Copy, Binary). + +copy_shared_term_2(Copy, Binary) -> + erlang:garbage_collect(), + BinCopy = binary_to_term(Binary), + test_eq(Copy, BinCopy), + ok. + +test_eq(A, B) -> + case A of + B -> ok; + _ -> test_eq_fail("FAILED MATCH", A, B) + end, + case A == B of + true -> ok; + false -> test_eq_fail("FAILED EQUALITY", A, B) + end. + +test_eq_fail(Error, A, B) -> + io:format("~s:\n\nA = ~p\n\nB = ~p\n", + [Error, printable(A), printable(B)]), + ct:fail(Error). + +rand_term(Size) -> + F = rand:uniform(100), % to produce non-literals + Big = 666_701_523_687_345_689_643 * F, + MagicRef = atomics:new(10,[]), + Leafs = {atom, 42, 42.17*F, + Big, -Big, + [], {}, #{}, + "literal cons", + {"literal boxed"}, + fun lists:sort/1, + fun() -> F end, + self(), + lists:last(erlang:ports()), + make_ref(), + MagicRef, + <<F:(8*10)>>, % HeapBin + <<F:(8*65)>>, % ProcBin + <<F:7>>, % SubBin + HeapBin + <<F:(8*80+1)>>, % SubBin + ProcBin + mk_ext_pid({a@b, 17}, 17, 42), + mk_ext_port({a@b, 21}, 13), + mk_ext_ref({a@b, 42}, [42, 19, 11])}, + rand_term(Leafs, Size). + +rand_term(Leafs, Arity) when Arity > 0 -> + Length = rand:uniform(Arity), + List = [rand_term(Leafs, Arity-Length) || _ <- lists:seq(1,Length)], + case rand:uniform(6) of + 1 -> List; + 2 -> list_to_improper_list(List); + 3 -> list_to_tuple(List); + 4 -> list_to_flatmap(List); + 5 -> list_to_hashmap(List); + 6 -> list_to_fun(List) + end; +rand_term(Leafs, 0) -> + element(rand:uniform(size(Leafs)), Leafs). + +list_to_improper_list([A,B|T]) -> + T ++ [A|B]; +list_to_improper_list([H]) -> + [[]|H]. + +list_to_flatmap(List) -> + list_to_map(List, #{}). + +list_to_hashmap(List) -> + HashMap = #{1=>1, 2=>2, 3=>3, 4=>4, 5=>5, 6=>6, 7=>7, 8=>8, 9=>9,10=>0, + 11=>1,12=>2,13=>3,14=>4,15=>5,16=>6,17=>7,18=>8,19=>9,20=>0, + 21=>1,22=>2,23=>3,24=>4,25=>5,26=>6,27=>7,28=>8,29=>9,30=>0, + 31=>1,32=>2,33=>3}, + list_to_map(List, HashMap). + +list_to_map([], Map) -> + Map; +list_to_map([K], Map) -> + Map#{K => K}; +list_to_map([K,V|T], Map) -> + list_to_map(T, Map#{K => V}). + +list_to_fun([X]) -> + fun(A) -> A + X end; +list_to_fun([X, Y]) -> + fun(A) -> A + X + Y end; +list_to_fun([X, Y | T]) -> + fun(A) -> [A+X+Y | T] end. + + +%% Convert local funs to maps to show fun environment +printable(Fun) when is_function(Fun) -> + case erlang:fun_info(Fun, type) of + {type,local} -> + {env, Env} = erlang:fun_info(Fun, env), + #{'fun' => [printable(T) || T <- Env]}; + {type,external} -> + Fun + end; +printable([H|T]) -> + [printable(H)|printable(T)]; +printable(Tuple) when is_tuple(Tuple) -> + list_to_tuple(printable(tuple_to_list(Tuple))); +printable(Map) when is_map(Map) -> + maps:from_list(printable(maps:to_list(Map))); +printable(Leaf) -> + Leaf. diff --git a/erts/emulator/test/exception_SUITE.erl b/erts/emulator/test/exception_SUITE.erl index 746993bf34..80fc35d2be 100644 --- a/erts/emulator/test/exception_SUITE.erl +++ b/erts/emulator/test/exception_SUITE.erl @@ -145,6 +145,12 @@ pending({badarg,[{erlang,Bif,BifArgs,Loc1}, when is_atom(Bif), is_list(BifArgs), length(Args) =:= Arity, is_list(Loc1), is_list(Loc2) -> ok; +pending({badarith,[{erlang,Bif,BifArgs,Loc1}, + {?MODULE,Func,Arity,Loc2}|_]}, + Func, Args, _Code) + when is_atom(Bif), is_list(BifArgs), length(Args) =:= Arity, + is_list(Loc1), is_list(Loc2) -> + ok; pending({undef,[{non_existing_module,foo,[],Loc}|_]}, _, _, _) when is_list(Loc) -> ok; @@ -1032,6 +1038,7 @@ error_info(_Config) -> {open_port, [{bad,name}, []]}, {open_port, [{spawn, "no_command"}, bad_option_list]}, {open_port, [{spawn, "no_command"}, [xyz]]}, + {open_port, [{spawn, "no_command"}, [{args,[]}]],[{1,".*spawn_executable.*"}]}, {port_call, 2}, %Internal BIF. @@ -1549,6 +1556,13 @@ line_numbers(Config) when is_list(Config) -> {?MODULE,test_tl,1,[{file,"list_bifs.erl"},{line,102}]}|_]}} = (catch test_tl(y)), + %% This line number is too large to be represented and will be silently + %% ignored by the emulator. + {'EXIT',{crash, + [{?MODULE,crash_huge_line,1,[]}, + {?MODULE,line_numbers,1,_}|_]}} = + (catch crash_huge_line(gurka)), + ok. id(I) -> I. @@ -1671,3 +1685,8 @@ update_map(M0) -> %Line 2 test_hd(X) -> foo(), hd(X). %Line 101 test_tl(X) -> foo(), tl(X). %Line 102 foo() -> id(100). + +-file("huge_lines.erl", 100000000). %Line 100000000 + +crash_huge_line(_) -> %Line 100000002 + erlang:error(crash). %Line 100000003 diff --git a/erts/emulator/test/monitor_SUITE.erl b/erts/emulator/test/monitor_SUITE.erl index 126fc98a7c..49fe788faf 100644 --- a/erts/emulator/test/monitor_SUITE.erl +++ b/erts/emulator/test/monitor_SUITE.erl @@ -28,7 +28,9 @@ demon_2/1, demon_3/1, demonitor_flush/1, gh_5225_demonitor_alias/1, local_remove_monitor/1, remote_remove_monitor/1, mon_1/1, mon_2/1, large_exit/1, list_cleanup/1, mixer/1, named_down/1, otp_5827/1, - monitor_time_offset/1, monitor_tag_storage/1]). + monitor_time_offset/1, monitor_tag_storage/1, + unexpected_alias_at_demonitor_gh5310/1, + down_on_alias_gh5310/1]). -export([y2/1, g/1, g0/0, g1/0, large_exit_sub/1]). @@ -41,7 +43,9 @@ all() -> demon_1, mon_1, mon_2, demon_2, demon_3, demonitor_flush, gh_5225_demonitor_alias, {group, remove_monitor}, large_exit, list_cleanup, mixer, named_down, otp_5827, - monitor_time_offset, monitor_tag_storage]. + monitor_time_offset, monitor_tag_storage, + unexpected_alias_at_demonitor_gh5310, + down_on_alias_gh5310]. groups() -> [{remove_monitor, [], @@ -946,12 +950,108 @@ receive_tagged_down_msgs(Tag, Msgs) -> Msgs end. +unexpected_alias_at_demonitor_gh5310(Config) when is_list(Config) -> + %% The demonitor operation erroneously behaved as if the + %% monitor had been created using the {alias, explicit_unalias} + %% option... + Pid = spawn_link(fun () -> + receive + {alias, Alias} -> + Alias ! {hello_via_alias, self()} + end + end), + Mon = erlang:monitor(process, Pid), + AliasMon = erlang:monitor(process, Pid, [{alias, reply_demonitor}]), + erlang:demonitor(AliasMon, [flush]), + Pid ! {alias, AliasMon}, + receive + {'DOWN', Mon, process, Pid, normal} -> + ok + end, + receive + {hello_via_alias, Pid} -> + ct:fail(unexpected_message_via_alias) + after + 0 -> + ok + end. + +down_on_alias_gh5310(Config) when is_list(Config) -> + %% Could only occur when the internal monitor structure was transformed + %% into an alias structure during the demonitor() operation, the target + %% terminated before the demonitor signal reached it, and the exit reason + %% wasn't an immediate. We test with both immediate and compound exit + %% reason just to make sure we don't introduce the bug in the immediate + %% case at a later time... + process_flag(scheduler, 1), + {DeMonSched, TermSched} = case erlang:system_info(schedulers) of + 1 -> {1, 1}; + 2 -> {1, 2}; + _ -> {2, 3} + end, + lists:foreach(fun (N) -> + ImmedExitReason = case N rem 2 of + 0 -> true; + _ -> false + end, + down_on_alias_gh5310_test(ImmedExitReason, + DeMonSched, TermSched) + end, + lists:seq(1, 200)), + ok. + +down_on_alias_gh5310_test(ImmedExitReason, DeMonSched, TermSched) -> + Go = make_ref(), + Done = make_ref(), + Parent = self(), + TermPid = spawn_opt(fun () -> + Parent ! {ready, self()}, + receive Go -> + if ImmedExitReason == true -> + exit(bye); + true -> + exit(Go) + end + end + end, [{scheduler, TermSched}]), + DeMonPid = spawn_opt(fun () -> + AliasMon = erlang:monitor(process, TermPid, + [{alias, explicit_unalias}]), + Parent ! {ready, self()}, + receive Go -> ok end, + erlang:demonitor(AliasMon, [flush]), + busy_wait_until(fun () -> + not is_process_alive(TermPid) + end), + receive + {'DOWN', AliasMon, process, _, _} = DownMsg -> + exit({unexpected_msg, DownMsg}) + after + 0 -> + Parent ! Done + end + end, [{scheduler, DeMonSched}, link]), + receive {ready, TermPid} -> ok end, + receive {ready, DeMonPid} -> ok end, + erlang:yield(), + TermPid ! Go, + DeMonPid ! Go, + receive Done -> ok end. + %% %% ... %% id(X) -> X. +busy_wait_until(Fun) -> + case catch Fun() of + true -> + ok; + _ -> + busy_wait_until(Fun) + end. + wait_until(Fun) -> case catch Fun() of true -> diff --git a/erts/emulator/test/nif_SUITE_data/nif_SUITE.c b/erts/emulator/test/nif_SUITE_data/nif_SUITE.c index b7580274ed..077161232a 100644 --- a/erts/emulator/test/nif_SUITE_data/nif_SUITE.c +++ b/erts/emulator/test/nif_SUITE_data/nif_SUITE.c @@ -2074,7 +2074,7 @@ static ERL_NIF_TERM call_nif_nan_or_inf(ErlNifEnv* env, int argc, const ERL_NIF_ assert(enif_is_exception(env, res)); assert(enif_has_pending_exception(env, NULL)); if (strcmp(arg, "tuple") == 0) { - return enif_make_tuple2(env, argv[0], res); + return enif_make_tuple2(env, argv[0], argv[0]); } else { return res; } diff --git a/erts/emulator/test/persistent_term_SUITE.erl b/erts/emulator/test/persistent_term_SUITE.erl index 3238807aba..0dad57e100 100644 --- a/erts/emulator/test/persistent_term_SUITE.erl +++ b/erts/emulator/test/persistent_term_SUITE.erl @@ -33,6 +33,7 @@ killed_while_trapping_erase/1, error_info/1, whole_message/1, + shared_magic_ref/1, non_message_signal/1]). %% @@ -55,6 +56,7 @@ all() -> killed_while_trapping_erase, error_info, whole_message, + shared_magic_ref, non_message_signal]. init_per_suite(Config) -> @@ -696,6 +698,19 @@ fck_loop_2(T, Key, MaxCollSz, MaxSzLeft0) -> +%% OTP-17700 Bug skipped refc++ of shared magic reference +shared_magic_ref(_Config) -> + Ref = atomics:new(10, []), + persistent_term:put(shared_magic_ref, {Ref, Ref}), + shared_magic_ref_cont(). + +shared_magic_ref_cont() -> + erlang:garbage_collect(), + {Ref, Ref} = persistent_term:get(shared_magic_ref), + 0 = atomics:get(Ref, 1), %% would definitely fail on debug vm + ok. + + %% Test that all persistent terms are erased by init:restart/0. init_restart(_Config) -> diff --git a/erts/emulator/test/signal_SUITE.erl b/erts/emulator/test/signal_SUITE.erl index 6d726289b6..a0d3e277c7 100644 --- a/erts/emulator/test/signal_SUITE.erl +++ b/erts/emulator/test/signal_SUITE.erl @@ -36,7 +36,12 @@ % Test cases -export([xm_sig_order/1, kill2killed/1, - contended_signal_handling/1]). + contended_signal_handling/1, + busy_dist_exit_signal/1, + busy_dist_demonitor_signal/1, + busy_dist_down_signal/1, + busy_dist_spawn_reply_signal/1, + busy_dist_unlink_ack_signal/1]). init_per_testcase(Func, Config) when is_atom(Func), is_list(Config) -> [{testcase, Func}|Config]. @@ -57,7 +62,12 @@ suite() -> all() -> [xm_sig_order, kill2killed, - contended_signal_handling]. + contended_signal_handling, + busy_dist_exit_signal, + busy_dist_demonitor_signal, + busy_dist_down_signal, + busy_dist_spawn_reply_signal, + busy_dist_unlink_ack_signal]. %% Test that exit signals and messages are received in correct order xm_sig_order(Config) when is_list(Config) -> @@ -209,6 +219,220 @@ contended_signal_handling_make_ports(Drv, N, Ports) -> true = is_port(Port), contended_signal_handling_make_ports(Drv, N-1, [Port|Ports]). +busy_dist_exit_signal(Config) when is_list(Config) -> + BusyTime = 1000, + {ok, BusyChannelNode} = start_node(Config), + {ok, OtherNode} = start_node(Config, "-proto_dist gen_tcp"), + Tester = self(), + Exiter = spawn(BusyChannelNode, + fun () -> + pong = net_adm:ping(OtherNode), + Tester ! {self(), alive}, + receive after infinity -> ok end + end), + receive {Exiter, alive} -> ok end, + Linker = spawn_link(OtherNode, + fun () -> + process_flag(trap_exit, true), + link(Exiter), + receive + {'EXIT', Exiter, Reason} -> + tester_killed_me = Reason, + Tester ! {self(), got_exiter_exit_message}; + Unexpected -> + exit({unexpected_message, Unexpected}) + end + end), + make_busy(BusyChannelNode, OtherNode, 1000), + exit(Exiter, tester_killed_me), + receive + {Linker, got_exiter_exit_message} -> + unlink(Linker), + ok + after + BusyTime*2 -> + ct:fail(missing_exit_signal) + end, + stop_node(BusyChannelNode), + stop_node(OtherNode), + ok. + +busy_dist_demonitor_signal(Config) when is_list(Config) -> + BusyTime = 1000, + {ok, BusyChannelNode} = start_node(Config), + {ok, OtherNode} = start_node(Config, "-proto_dist gen_tcp"), + Tester = self(), + Demonitorer = spawn(BusyChannelNode, + fun () -> + pong = net_adm:ping(OtherNode), + Tester ! {self(), alive}, + receive + {Tester, monitor, Pid} -> + _Mon = erlang:monitor(process, Pid) + end, + receive after infinity -> ok end + end), + receive {Demonitorer, alive} -> ok end, + Demonitoree = spawn_link(OtherNode, + fun () -> + wait_until(fun () -> + {monitored_by, MB1} + = process_info(self(), + monitored_by), + lists:member(Demonitorer, MB1) + end), + Tester ! {self(), monitored}, + wait_until(fun () -> + {monitored_by, MB2} + = process_info(self(), + monitored_by), + not lists:member(Demonitorer, MB2) + end), + Tester ! {self(), got_demonitorer_demonitor_signal} + end), + Demonitorer ! {self(), monitor, Demonitoree}, + receive {Demonitoree, monitored} -> ok end, + make_busy(BusyChannelNode, OtherNode, 1000), + exit(Demonitorer, tester_killed_me), + receive + {Demonitoree, got_demonitorer_demonitor_signal} -> + unlink(Demonitoree), + ok + after + BusyTime*2 -> + ct:fail(missing_demonitor_signal) + end, + stop_node(BusyChannelNode), + stop_node(OtherNode), + ok. + +busy_dist_down_signal(Config) when is_list(Config) -> + BusyTime = 1000, + {ok, BusyChannelNode} = start_node(Config), + {ok, OtherNode} = start_node(Config, "-proto_dist gen_tcp"), + Tester = self(), + Exiter = spawn(BusyChannelNode, + fun () -> + pong = net_adm:ping(OtherNode), + Tester ! {self(), alive}, + receive after infinity -> ok end + end), + receive {Exiter, alive} -> ok end, + Monitorer = spawn_link(OtherNode, + fun () -> + process_flag(trap_exit, true), + Mon = erlang:monitor(process, Exiter), + receive + {'DOWN', Mon, process, Exiter, Reason} -> + tester_killed_me = Reason, + Tester ! {self(), got_exiter_down_message}; + Unexpected -> + exit({unexpected_message, Unexpected}) + end + end), + make_busy(BusyChannelNode, OtherNode, 1000), + exit(Exiter, tester_killed_me), + receive + {Monitorer, got_exiter_down_message} -> + unlink(Monitorer), + ok + after + BusyTime*2 -> + ct:fail(missing_down_signal) + end, + stop_node(BusyChannelNode), + stop_node(OtherNode), + ok. + +busy_dist_spawn_reply_signal(Config) when is_list(Config) -> + BusyTime = 1000, + {ok, BusyChannelNode} = start_node(Config), + {ok, OtherNode} = start_node(Config, "-proto_dist gen_tcp"), + Tester = self(), + Spawner = spawn_link(OtherNode, + fun () -> + pong = net_adm:ping(BusyChannelNode), + Tester ! {self(), ready}, + receive {Tester, go} -> ok end, + ReqID = spawn_request(BusyChannelNode, + fun () -> ok end, + []), + receive + {spawn_reply, ReqID, Result, _Pid} -> + ok = Result, + Tester ! {self(), got_spawn_reply_message}; + Unexpected -> + exit({unexpected_message, Unexpected}) + end + end), + receive {Spawner, ready} -> ok end, + make_busy(BusyChannelNode, OtherNode, 1000), + Spawner ! {self(), go}, + receive + {Spawner, got_spawn_reply_message} -> + unlink(Spawner), + ok + after + BusyTime*2 -> + ct:fail(missing_spawn_reply_signal) + end, + stop_node(BusyChannelNode), + stop_node(OtherNode), + ok. + +-record(erl_link, {type, % process | port | dist_process + pid = [], + state, % linked | unlinking + id}). + +busy_dist_unlink_ack_signal(Config) when is_list(Config) -> + BusyTime = 1000, + {ok, BusyChannelNode} = start_node(Config), + {ok, OtherNode} = start_node(Config, "-proto_dist gen_tcp"), + Tester = self(), + Unlinkee = spawn(BusyChannelNode, + fun () -> + pong = net_adm:ping(OtherNode), + Tester ! {self(), alive}, + receive after infinity -> ok end + end), + receive {Unlinkee, alive} -> ok end, + Unlinker = spawn_link(OtherNode, + fun () -> + erts_debug:set_internal_state(available_internal_state, true), + link(Unlinkee), + #erl_link{type = dist_process, + pid = Unlinkee, + state = linked} = find_proc_link(self(), + Unlinkee), + Tester ! {self(), ready}, + receive {Tester, go} -> ok end, + unlink(Unlinkee), + #erl_link{type = dist_process, + pid = Unlinkee, + state = unlinking} = find_proc_link(self(), + Unlinkee), + wait_until(fun () -> + false == find_proc_link(self(), + Unlinkee) + end), + Tester ! {self(), got_unlink_ack_signal} + end), + receive {Unlinker, ready} -> ok end, + make_busy(BusyChannelNode, OtherNode, 1000), + Unlinker ! {self(), go}, + receive + {Unlinker, got_unlink_ack_signal} -> + unlink(Unlinker), + ok + after + BusyTime*2 -> + ct:fail(missing_unlink_ack_signal) + end, + stop_node(BusyChannelNode), + stop_node(OtherNode), + ok. + %% %% -- Internal utils -------------------------------------------------------- %% @@ -232,19 +456,116 @@ wait_until(Fun) -> wait_until(Fun) end. +find_proc_link(Pid, To) when is_pid(Pid), is_pid(To) -> + lists:keyfind(To, + #erl_link.pid, + erts_debug:get_internal_state({link_list, Pid})). + +make_busy(OnNode, ToNode, Time) -> + Parent = self(), + Fun = fun () -> + Proxy = self(), + Sspndr = spawn_link( + ToNode, + fun () -> + IC = find_gen_tcp_input_cntrlr(OnNode), + erlang:suspend_process(IC), + Proxy ! {self(), input_cntrlr_suspended}, + receive + {Proxy, resume_input_cntrlr} -> + erlang:resume_process(IC) + end, + Proxy ! {self(), input_cntrlr_resumed} + end), + receive + {Sspndr, input_cntrlr_suspended} -> + ok + end, + Spammer = spawn_link( + OnNode, + fun () -> + spammed = spam(ToNode), + Proxy ! {self(), channel_busy}, + receive + after Time -> ok + end, + Proxy ! {self(), timeout} + end), + receive + {Spammer, channel_busy} -> + Parent ! {self(), channel_busy} + end, + receive + {Spammer, timeout} -> + Sspndr ! {self(), resume_input_cntrlr} + end, + receive + {Sspndr, input_cntrlr_resumed} -> + ok + end + end, + Proxy = spawn_link(Fun), + receive + {Proxy, channel_busy} -> + ok + end, + Proxy. + +find_gen_tcp_input_cntrlr(Node) when is_atom(Node) -> + case lists:keyfind(Node, 1, erlang:system_info(dist_ctrl)) of + {Node, DistCtrl} -> + find_gen_tcp_input_cntrlr(DistCtrl); + false -> + undefined + end; +find_gen_tcp_input_cntrlr(DistCtrl) when is_pid(DistCtrl) -> + {links, LList} = process_info(DistCtrl, links), + try + lists:foreach(fun (Pid) -> + case process_info(Pid, initial_call) of + {initial_call, + {gen_tcp_dist,dist_cntrlr_input_setup,3}} -> + throw({input_ctrlr, Pid}); + _ -> + ok + end + end, + LList), + undefined + catch + throw:{input_ctrlr, DistInputCtrlr} -> + DistInputCtrlr + end. + +spam(Node) -> + To = {'__a_name_hopefully_not_registered__', Node}, + Data = lists:seq(1, 100), + spam(To, Data). + +spam(To, Data) -> + case erlang:send(To, Data, [nosuspend]) of + nosuspend -> + spammed; + _ -> + spam(To, Data) + end. + repeat(_Fun, N) when is_integer(N), N =< 0 -> ok; repeat(Fun, N) when is_integer(N) -> Fun(), repeat(Fun, N-1). -start_node(Config) -> +start_node(Config, Args) -> Name = list_to_atom(atom_to_list(?MODULE) ++ "-" ++ atom_to_list(proplists:get_value(testcase, Config)) ++ "-" ++ integer_to_list(erlang:system_time(second)) ++ "-" ++ integer_to_list(erlang:unique_integer([positive]))), Pa = filename:dirname(code:which(?MODULE)), - test_server:start_node(Name, slave, [{args, "-pa " ++ Pa}]). + test_server:start_node(Name, slave, [{args, "-pa " ++ Pa ++ " " ++ Args}]). + +start_node(Config) -> + start_node(Config, ""). stop_node(Node) -> test_server:stop_node(Node). diff --git a/erts/emulator/test/timer_bif_SUITE.erl b/erts/emulator/test/timer_bif_SUITE.erl index 6e5b672646..bd3446914c 100644 --- a/erts/emulator/test/timer_bif_SUITE.erl +++ b/erts/emulator/test/timer_bif_SUITE.erl @@ -897,7 +897,7 @@ mem_get() -> % Bif timer memory Ref = make_ref(), erlang:system_info({memory_internal, Ref, [fix_alloc]}), - mem_recv(erlang:system_info(schedulers), Ref, {0, 0}). + mem_recv(erts_internal:no_aux_work_threads()-1, Ref, {0, 0}). mem_recv(0, _Ref, AU) -> AU; diff --git a/erts/epmd/src/epmd_srv.c b/erts/epmd/src/epmd_srv.c index 0cb6f48b7f..e10a6c4b95 100644 --- a/erts/epmd/src/epmd_srv.c +++ b/erts/epmd/src/epmd_srv.c @@ -212,7 +212,7 @@ static const char *epmd_ntop(struct sockaddr_storage *sa, char *buff, size_t len /* Save errno so that it is not changed by inet_ntop */ int myerrno = errno; const char *res; -#if !defined(EPMD6) +#if defined(EPMD6) if (sa->ss_family == AF_INET6) { struct sockaddr_in6 *addr = (struct sockaddr_in6 *)sa; res = inet_ntop( @@ -458,7 +458,7 @@ void run(EpmdVars *g) if (fcntl(listensock[i], F_SETFL, opt | O_NONBLOCK) == -1) #endif /* __WIN32__ */ dbg_perror(g,"failed to set non-blocking mode of listening socket %d on ipaddr %s", - listensock[i], epmd_ntop(&iserv_addr[num_sockets], + listensock[i], epmd_ntop(&iserv_addr[i], socknamebuf, sizeof(socknamebuf))); if (bind(listensock[i], sa, salen) < 0) @@ -466,14 +466,14 @@ void run(EpmdVars *g) if (errno == EADDRINUSE) { dbg_tty_printf(g,1,"there is already a epmd running at port %d on ipaddr %s", - g->port, epmd_ntop(&iserv_addr[num_sockets], + g->port, epmd_ntop(&iserv_addr[i], socknamebuf, sizeof(socknamebuf))); epmd_cleanup_exit(g,0); } else { dbg_perror(g,"failed to bind on ipaddr %s", - epmd_ntop(&iserv_addr[num_sockets], + epmd_ntop(&iserv_addr[i], socknamebuf, sizeof(socknamebuf))); epmd_cleanup_exit(g,1); } @@ -481,7 +481,7 @@ void run(EpmdVars *g) if(listen(listensock[i], SOMAXCONN) < 0) { dbg_perror(g,"failed to listen on ipaddr %s", - epmd_ntop(&iserv_addr[num_sockets], + epmd_ntop(&iserv_addr[i], socknamebuf, sizeof(socknamebuf))); epmd_cleanup_exit(g,1); } diff --git a/erts/etc/common/Makefile.in b/erts/etc/common/Makefile.in index ba4d898f79..3460b19f38 100644 --- a/erts/etc/common/Makefile.in +++ b/erts/etc/common/Makefile.in @@ -319,9 +319,34 @@ $(LOGMESS_GENERATED): $(WINETC)/erlsrv/erlsrv_logmess.mc $(V_MC) -o $(OBJDIR) $(WINETC)/erlsrv/erlsrv_logmess.mc && \ echo $? >$(LOGMESS_GENERATED) +# +# Make version fit into max 4 integers separated by comma. +# +COMMA_CHAR := , +SPACE_CHAR := $(subst ,, ) + +OTP_VSN_COMMA_SEP := $(shell echo $(OTP_VERSION) | sed -e 's;-[^-]\+$$;;' ) + +ERTS_VSN_COMMA_SEP := $(subst .,$(SPACE_CHAR),$(VSN)) +OTP_VSN_COMMA_SEP := $(subst .,$(SPACE_CHAR),$(OTP_VSN_COMMA_SEP)) + +ERTS_VSN_COMMA_SEP := $(wordlist 1,4, $(ERTS_VSN_COMMA_SEP)) +OTP_VSN_COMMA_SEP := $(wordlist 1,4, $(OTP_VSN_COMMA_SEP)) + +ERTS_VSN_COMMA_SEP := $(subst $(SPACE_CHAR),$(COMMA_CHAR),$(ERTS_VSN_COMMA_SEP)) +OTP_VSN_COMMA_SEP := $(subst $(SPACE_CHAR),$(COMMA_CHAR),$(OTP_VSN_COMMA_SEP)) + +$(WINETC)/version.h: $(WINETC)/version.h.src + $(vsn_verbose)sed -e 's;%ERTS_VERSION%;$(VSN);' \ + -e 's;%ERTS_VERSION_INTS%;$(ERTS_VSN_COMMA_SEP);' \ + -e 's;%OTP_VERSION%;$(OTP_VERSION);' \ + -e 's;%OTP_VERSION_INTS%;$(OTP_VSN_COMMA_SEP);' \ + $(WINETC)/version.h.src > $(WINETC)/version.h + $(OBJDIR)/$(ERLRES_OBJ): $(WINETC)/erl.rc $(WINETC)/erlang.ico \ $(WINETC)/erl_icon.ico $(WINETC)/hrl_icon.ico \ - $(WINETC)/beam_icon.ico $(LOGMESS_GENERATED) + $(WINETC)/beam_icon.ico $(LOGMESS_GENERATED) \ + $(WINETC)/version.h $(V_RC) -o $@ -I$(WINETC) $(WINETC)/erl.rc ifeq ($(USING_VC), yes) diff --git a/erts/etc/common/erlexec.c b/erts/etc/common/erlexec.c index 06182c25ce..e5f8ee6f77 100644 --- a/erts/etc/common/erlexec.c +++ b/erts/etc/common/erlexec.c @@ -119,6 +119,7 @@ static char *plusM_other_switches[] = { "Ytt", "Iscs", "Xscs", + "dai", NULL }; diff --git a/erts/etc/darwin/Info.plist b/erts/etc/darwin/Info.plist index a12bcd9d68..3896524588 100644 --- a/erts/etc/darwin/Info.plist +++ b/erts/etc/darwin/Info.plist @@ -2,14 +2,6 @@ <!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd"> <plist version="1.0"> <dict> - <key>CFBundleName</key> - <string>Erlang</string> - <key>CFBundleDisplayname</key> - <string>Erlang</string> - <key>CFBundleDevelopmentRegion</key> - <string>English</string> - <key>CFBundleIdentifier</key> - <string>org.erlang.beam</string> <key>NSAppTransportSecurity</key> <dict> <key>NSExceptionDomains</key> diff --git a/erts/etc/win32/erl.rc b/erts/etc/win32/erl.rc index 772213ac86..3ff41b8eaa 100644 --- a/erts/etc/win32/erl.rc +++ b/erts/etc/win32/erl.rc @@ -19,12 +19,31 @@ // #include <windows.h> #include "resource.h" +#include "version.h" 1 ICON DISCARDABLE "erlang.ico" - - - +VS_VERSION_INFO VERSIONINFO + FILEVERSION ERTS_VERSION_INTS + PRODUCTVERSION OTP_VERSION_INTS +{ + BLOCK "StringFileInfo" + { + BLOCK "040904e4" + { + VALUE "CompanyName", "Ericsson\0" + VALUE "FileDescription", "Erlang\0" + VALUE "FileVersion", ERTS_VERSION_STR + VALUE "LegalCopyright", "Ericsson. All rights reserved.\0" + VALUE "ProductName", "Erlang/OTP\0" + VALUE "ProductVersion", OTP_VERSION_STR + } + } + BLOCK "VarFileInfo" + { + VALUE "Translation", 0x0409, 1252 + } +} diff --git a/erts/etc/win32/version.h.src b/erts/etc/win32/version.h.src new file mode 100644 index 0000000000..266ebda47d --- /dev/null +++ b/erts/etc/win32/version.h.src @@ -0,0 +1,7 @@ + + +#define ERTS_VERSION_INTS %ERTS_VERSION_INTS% +#define ERTS_VERSION_STR "%ERTS_VERSION%\0" + +#define OTP_VERSION_INTS %OTP_VERSION_INTS% +#define OTP_VERSION_STR "%OTP_VERSION%\0" diff --git a/erts/include/internal/gcc/ethr_atomic.h b/erts/include/internal/gcc/ethr_atomic.h index 3ef042ce61..f57613bc2b 100644 --- a/erts/include/internal/gcc/ethr_atomic.h +++ b/erts/include/internal/gcc/ethr_atomic.h @@ -496,13 +496,13 @@ ETHR_NATMC_FUNC__(or_retold_mb)(ETHR_ATMC_T__ *var, ETHR_AINT_T__ mask) static ETHR_INLINE ETHR_AINT_T__ ETHR_NATMC_FUNC__(cmpxchg)(ETHR_ATMC_T__ *var, - ETHR_AINT_T__ new, + ETHR_AINT_T__ new_val, ETHR_AINT_T__ exp) { ETHR_AINT_T__ xchg = exp; if (__atomic_compare_exchange_n(&var->value, &xchg, - new, + new_val, 0, /* No spurious failures, please */ __ATOMIC_RELAXED, __ATOMIC_RELAXED)) @@ -522,13 +522,13 @@ ETHR_NATMC_FUNC__(cmpxchg)(ETHR_ATMC_T__ *var, static ETHR_INLINE ETHR_AINT_T__ ETHR_NATMC_FUNC__(cmpxchg_acqb)(ETHR_ATMC_T__ *var, - ETHR_AINT_T__ new, + ETHR_AINT_T__ new_val, ETHR_AINT_T__ exp) { ETHR_AINT_T__ xchg = exp; if (__atomic_compare_exchange_n(&var->value, &xchg, - new, + new_val, 0, /* No spurious failures, please */ __ATOMIC_ACQUIRE, __ATOMIC_ACQUIRE)) @@ -551,10 +551,10 @@ ETHR_NATMC_FUNC__(cmpxchg_acqb)(ETHR_ATMC_T__ *var, static ETHR_INLINE ETHR_AINT_T__ ETHR_NATMC_FUNC__(cmpxchg_mb)(ETHR_ATMC_T__ *var, - ETHR_AINT_T__ new, + ETHR_AINT_T__ new_val, ETHR_AINT_T__ old) { - return __sync_val_compare_and_swap(&var->value, old, new); + return __sync_val_compare_and_swap(&var->value, old, new_val); } #endif /* ETHR_HAVE___sync_val_compare_and_swap */ diff --git a/erts/include/internal/gcc/ethr_dw_atomic.h b/erts/include/internal/gcc/ethr_dw_atomic.h index 69cf6cd3f3..73409230c6 100644 --- a/erts/include/internal/gcc/ethr_dw_atomic.h +++ b/erts/include/internal/gcc/ethr_dw_atomic.h @@ -178,7 +178,7 @@ ethr_native_su_dw_atomic_read_acqb(ethr_native_dw_atomic_t *var) static ETHR_INLINE ETHR_NATIVE_SU_DW_SINT_T ethr_native_su_dw_atomic_cmpxchg(ethr_native_dw_atomic_t *var, - ETHR_NATIVE_SU_DW_SINT_T new, + ETHR_NATIVE_SU_DW_SINT_T new_val, ETHR_NATIVE_SU_DW_SINT_T exp) { ethr_native_dw_ptr_t p = (ethr_native_dw_ptr_t) ETHR_DW_NATMC_MEM__(var); @@ -186,7 +186,7 @@ ethr_native_su_dw_atomic_cmpxchg(ethr_native_dw_atomic_t *var, ETHR_DW_DBG_ALIGNED__(p); if (__atomic_compare_exchange_n(p, &xchg, - new, + new_val, 0, __ATOMIC_RELAXED, __ATOMIC_RELAXED)) @@ -202,7 +202,7 @@ ethr_native_su_dw_atomic_cmpxchg(ethr_native_dw_atomic_t *var, static ETHR_INLINE ETHR_NATIVE_SU_DW_SINT_T ethr_native_su_dw_atomic_cmpxchg_acqb(ethr_native_dw_atomic_t *var, - ETHR_NATIVE_SU_DW_SINT_T new, + ETHR_NATIVE_SU_DW_SINT_T new_val, ETHR_NATIVE_SU_DW_SINT_T exp) { ethr_native_dw_ptr_t p = (ethr_native_dw_ptr_t) ETHR_DW_NATMC_MEM__(var); @@ -210,7 +210,7 @@ ethr_native_su_dw_atomic_cmpxchg_acqb(ethr_native_dw_atomic_t *var, ETHR_DW_DBG_ALIGNED__(p); if (__atomic_compare_exchange_n(p, &xchg, - new, + new_val, 0, __ATOMIC_ACQUIRE, __ATOMIC_ACQUIRE)) @@ -229,12 +229,12 @@ ethr_native_su_dw_atomic_cmpxchg_acqb(ethr_native_dw_atomic_t *var, static ETHR_INLINE ETHR_NATIVE_SU_DW_SINT_T ethr_native_su_dw_atomic_cmpxchg_mb(ethr_native_dw_atomic_t *var, - ETHR_NATIVE_SU_DW_SINT_T new, + ETHR_NATIVE_SU_DW_SINT_T new_val, ETHR_NATIVE_SU_DW_SINT_T old) { ethr_native_dw_ptr_t p = (ethr_native_dw_ptr_t) ETHR_DW_NATMC_MEM__(var); ETHR_DW_DBG_ALIGNED__(p); - return __sync_val_compare_and_swap(p, old, new); + return __sync_val_compare_and_swap(p, old, new_val); } #endif /* ETHR_HAVE___sync_val_compare_and_swap */ diff --git a/erts/lib_src/pthread/ethr_event.c b/erts/lib_src/pthread/ethr_event.c index 9d6e26fd81..57450448c4 100644 --- a/erts/lib_src/pthread/ethr_event.c +++ b/erts/lib_src/pthread/ethr_event.c @@ -36,6 +36,12 @@ # define _DARWIN_UNLIMITED_SELECT #endif +/* + * According to posix, select() implementations should + * support a max timeout value of at least 31 days. + */ +#define ETHR_SELECT_MAX_TV_SEC__ (31*24*60*60-1) + #include "ethread.h" #undef ETHR_INCLUDE_MONOTONIC_CLOCK__ #define ETHR_INCLUDE_MONOTONIC_CLOCK__ @@ -500,6 +506,7 @@ wait__(ethr_event *e, int spincount, ethr_sint64_t timeout) #endif fd_set *rsetp, *esetp; struct timeval select_timeout; + int select_timeout_res; #ifdef ETHR_HAVE_ETHR_GET_MONOTONIC_TIME #if ERTS_USE_PREMATURE_TIMEOUT @@ -527,7 +534,22 @@ wait__(ethr_event *e, int spincount, ethr_sint64_t timeout) #endif select_timeout.tv_sec = time / (1000*1000); - select_timeout.tv_usec = time % (1000*1000); + + if (select_timeout.tv_sec <= ETHR_SELECT_MAX_TV_SEC__) { + select_timeout.tv_usec = time % (1000*1000); + select_timeout_res = ETIMEDOUT; + } + else { + select_timeout.tv_sec = ETHR_SELECT_MAX_TV_SEC__; + select_timeout.tv_usec = 0; + /* + * Return EINTR (spurious wakeup) instead of + * ETIMEDOUT if we time out on this (huge) + * timeout value. Caller is responsible for + * restarting the wait... + */ + select_timeout_res = EINTR; + } ETHR_ASSERT(val != ETHR_EVENT_ON__); @@ -577,7 +599,7 @@ wait__(ethr_event *e, int spincount, ethr_sint64_t timeout) sres = select(fd + 1, rsetp, NULL, esetp, &select_timeout); if (sres == 0) - res = ETIMEDOUT; + res = select_timeout_res; else { res = EINTR; if (sres < 0 && errno != EINTR) diff --git a/erts/lib_src/yielding_c_fun/.gitignore b/erts/lib_src/yielding_c_fun/.gitignore index eb74315e49..d50bdd2744 100644 --- a/erts/lib_src/yielding_c_fun/.gitignore +++ b/erts/lib_src/yielding_c_fun/.gitignore @@ -1,6 +1,7 @@ *.o *~ bin/yielding_c_fun.bin +bin/win32 core test/test_trap_out.c GPATH diff --git a/erts/lib_src/yielding_c_fun/ycf_yield_fun.c b/erts/lib_src/yielding_c_fun/ycf_yield_fun.c index 1b9836bd4a..37bb3b385b 100644 --- a/erts/lib_src/yielding_c_fun/ycf_yield_fun.c +++ b/erts/lib_src/yielding_c_fun/ycf_yield_fun.c @@ -909,6 +909,7 @@ void ast_add_yield_code_generated_define(ycf_node* source_out_tree/*Will be chan "static void* ycf_find_stack_bottom_conservative_helper() {\n" " void* p = NULL;\n" " volatile intptr_t tmp = (intptr_t)&p;\n" + " /* codechecker_intentional [StackAddrEscapeBase] */\n" " return (void*)tmp;\n" "}\n" "static void* ycf_find_stack_bottom_conservative() {\n" diff --git a/erts/preloaded/ebin/atomics.beam b/erts/preloaded/ebin/atomics.beam Binary files differindex 3b59aefa80..485d5c0d42 100644 --- a/erts/preloaded/ebin/atomics.beam +++ b/erts/preloaded/ebin/atomics.beam diff --git a/erts/preloaded/ebin/counters.beam b/erts/preloaded/ebin/counters.beam Binary files differindex f6b3ee4236..b8e8b665f5 100644 --- a/erts/preloaded/ebin/counters.beam +++ b/erts/preloaded/ebin/counters.beam diff --git a/erts/preloaded/ebin/erl_init.beam b/erts/preloaded/ebin/erl_init.beam Binary files differindex 678ff35564..e5db970515 100644 --- a/erts/preloaded/ebin/erl_init.beam +++ b/erts/preloaded/ebin/erl_init.beam diff --git a/erts/preloaded/ebin/erl_prim_loader.beam b/erts/preloaded/ebin/erl_prim_loader.beam Binary files differindex 003d7f8d98..b597e7c21c 100644 --- a/erts/preloaded/ebin/erl_prim_loader.beam +++ b/erts/preloaded/ebin/erl_prim_loader.beam diff --git a/erts/preloaded/ebin/erl_tracer.beam b/erts/preloaded/ebin/erl_tracer.beam Binary files differindex 35fb1c1df5..eee28c574f 100644 --- a/erts/preloaded/ebin/erl_tracer.beam +++ b/erts/preloaded/ebin/erl_tracer.beam diff --git a/erts/preloaded/ebin/erlang.beam b/erts/preloaded/ebin/erlang.beam Binary files differindex 421c4803f9..fc7e58c9f9 100644 --- a/erts/preloaded/ebin/erlang.beam +++ b/erts/preloaded/ebin/erlang.beam diff --git a/erts/preloaded/ebin/erts_code_purger.beam b/erts/preloaded/ebin/erts_code_purger.beam Binary files differindex 8e22c47267..3b87631690 100644 --- a/erts/preloaded/ebin/erts_code_purger.beam +++ b/erts/preloaded/ebin/erts_code_purger.beam diff --git a/erts/preloaded/ebin/erts_dirty_process_signal_handler.beam b/erts/preloaded/ebin/erts_dirty_process_signal_handler.beam Binary files differindex 0e0cbf0ad5..962cd21eb6 100644 --- a/erts/preloaded/ebin/erts_dirty_process_signal_handler.beam +++ b/erts/preloaded/ebin/erts_dirty_process_signal_handler.beam diff --git a/erts/preloaded/ebin/erts_internal.beam b/erts/preloaded/ebin/erts_internal.beam Binary files differindex 540be7491e..a7c464ad71 100644 --- a/erts/preloaded/ebin/erts_internal.beam +++ b/erts/preloaded/ebin/erts_internal.beam diff --git a/erts/preloaded/ebin/erts_literal_area_collector.beam b/erts/preloaded/ebin/erts_literal_area_collector.beam Binary files differindex 580ac9a107..effb1d5190 100644 --- a/erts/preloaded/ebin/erts_literal_area_collector.beam +++ b/erts/preloaded/ebin/erts_literal_area_collector.beam diff --git a/erts/preloaded/ebin/init.beam b/erts/preloaded/ebin/init.beam Binary files differindex 2066108897..154038aa21 100644 --- a/erts/preloaded/ebin/init.beam +++ b/erts/preloaded/ebin/init.beam diff --git a/erts/preloaded/ebin/persistent_term.beam b/erts/preloaded/ebin/persistent_term.beam Binary files differindex 22151a59e5..95e9c74faa 100644 --- a/erts/preloaded/ebin/persistent_term.beam +++ b/erts/preloaded/ebin/persistent_term.beam diff --git a/erts/preloaded/ebin/prim_buffer.beam b/erts/preloaded/ebin/prim_buffer.beam Binary files differindex 0acb67d9e5..50bb6b1a64 100644 --- a/erts/preloaded/ebin/prim_buffer.beam +++ b/erts/preloaded/ebin/prim_buffer.beam diff --git a/erts/preloaded/ebin/prim_eval.beam b/erts/preloaded/ebin/prim_eval.beam Binary files differindex db5dbdf9f5..c8092d33f3 100644 --- a/erts/preloaded/ebin/prim_eval.beam +++ b/erts/preloaded/ebin/prim_eval.beam diff --git a/erts/preloaded/ebin/prim_file.beam b/erts/preloaded/ebin/prim_file.beam Binary files differindex c50d824325..3e4c868b74 100644 --- a/erts/preloaded/ebin/prim_file.beam +++ b/erts/preloaded/ebin/prim_file.beam diff --git a/erts/preloaded/ebin/prim_inet.beam b/erts/preloaded/ebin/prim_inet.beam Binary files differindex cebf181e14..25c07335d4 100644 --- a/erts/preloaded/ebin/prim_inet.beam +++ b/erts/preloaded/ebin/prim_inet.beam diff --git a/erts/preloaded/ebin/prim_net.beam b/erts/preloaded/ebin/prim_net.beam Binary files differindex e35894f550..eaa3a7b74f 100644 --- a/erts/preloaded/ebin/prim_net.beam +++ b/erts/preloaded/ebin/prim_net.beam diff --git a/erts/preloaded/ebin/prim_socket.beam b/erts/preloaded/ebin/prim_socket.beam Binary files differindex 3e6e718e83..1a940bd8b3 100644 --- a/erts/preloaded/ebin/prim_socket.beam +++ b/erts/preloaded/ebin/prim_socket.beam diff --git a/erts/preloaded/ebin/prim_zip.beam b/erts/preloaded/ebin/prim_zip.beam Binary files differindex 668243a1f3..4091a13445 100644 --- a/erts/preloaded/ebin/prim_zip.beam +++ b/erts/preloaded/ebin/prim_zip.beam diff --git a/erts/preloaded/ebin/socket_registry.beam b/erts/preloaded/ebin/socket_registry.beam Binary files differindex 49e3c75798..a08fff3b2d 100644 --- a/erts/preloaded/ebin/socket_registry.beam +++ b/erts/preloaded/ebin/socket_registry.beam diff --git a/erts/preloaded/ebin/zlib.beam b/erts/preloaded/ebin/zlib.beam Binary files differindex 397f28c8c7..49b933c478 100644 --- a/erts/preloaded/ebin/zlib.beam +++ b/erts/preloaded/ebin/zlib.beam diff --git a/erts/preloaded/src/erlang.erl b/erts/preloaded/src/erlang.erl index 0c43b690a0..da87ef1c46 100644 --- a/erts/preloaded/src/erlang.erl +++ b/erts/preloaded/src/erlang.erl @@ -74,7 +74,65 @@ %% the correct function. -compile({inline, [badarg_with_info/1,error_with_info/2, error_with_inherited_info/3,badarg_with_cause/2]}). - +-compile(no_auto_import_types). + +%% Built-in datatypes +-type any() :: any(). +-type arity() :: arity(). +-type atom() :: atom(). +-type binary() :: <<_:_*8>>. +-type bitstring() :: <<_:_*1>>. +-type bool() :: boolean(). +-type boolean() :: true | false. +-type byte() :: 0..255. +-type char() :: 0..16#10FFFF. +-type float() :: float(). +-type function() :: fun(). +-type identifier() :: pid() | port() | reference(). +-type integer() :: integer(). +-type iodata() :: iolist() | binary(). +-type iolist() :: maybe_improper_list(byte() | binary() | iolist(), binary() | []). +-type list() :: list(). +-type list(ContentType) :: list(ContentType). +-type map() :: #{ any() => any() }. +-type maybe_improper_list() :: maybe_improper_list(any(), any()). +-type maybe_improper_list(ContentType, TerminationType) :: maybe_improper_list(ContentType, TerminationType). +-type mfa() :: {module(),atom(),arity()}. +-type module() :: atom(). +-type neg_integer() :: neg_integer(). +-type nil() :: []. +-type no_return() :: none(). +-type node() :: atom(). +-type non_neg_integer() :: non_neg_integer(). +-type none() :: none(). +-type nonempty_binary() :: <<_:8, _:_*8>>. +-type nonempty_bitstring() :: <<_:1, _:_*1>>. +-type nonempty_improper_list(ContentType, TerminationType) :: nonempty_improper_list(ContentType, TerminationType). +-type nonempty_list() :: nonempty_list(any()). +-type nonempty_list(ContentType) :: [ContentType, ...]. +-type nonempty_maybe_improper_list() :: nonempty_maybe_improper_list(any(), any()). +-type nonempty_maybe_improper_list(ContentType, TerminationType) :: nonempty_maybe_improper_list(ContentType, TerminationType). +-type nonempty_string() :: nonempty_list(char()). +-type number() :: integer() | float(). +-type pid() :: pid(). +-type port() :: port(). +-type pos_integer() :: pos_integer(). +-type reference() :: reference(). +-type string() :: [char()]. +-type term() :: any(). +-type timeout() :: 'infinity' | non_neg_integer(). +-type tuple() :: tuple(). +-export_type([any/0, arity/0, atom/0, binary/0, bitstring/0, bool/0, boolean/0, byte/0, + char/0, float/0, function/0, identifier/0, integer/0, iodata/0, iolist/0, + list/0, list/1, map/0, maybe_improper_list/0, maybe_improper_list/2, mfa/0, + module/0, neg_integer/0, nil/0, no_return/0, node/0, non_neg_integer/0, + none/0, nonempty_binary/0, nonempty_bitstring/0, nonempty_improper_list/2, + nonempty_list/0, nonempty_list/1, nonempty_maybe_improper_list/0, + nonempty_maybe_improper_list/2, nonempty_string/0, number/0, pid/0, + port/0, pos_integer/0, reference/0, string/0, term/0, timeout/0, + tuple/0]). + +%% Datatypes that need an erlang: prefix -export_type([timestamp/0]). -export_type([time_unit/0]). -export_type([deprecated_time_unit/0]). @@ -4185,7 +4243,7 @@ receive_emd(Ref, EMD, N) -> end. receive_emd(Ref) -> - receive_emd(Ref, #memory{}, erlang:system_info(schedulers)). + receive_emd(Ref, #memory{}, erts_internal:no_aux_work_threads()-1). aa_mem_data(#memory{total = Tot} = Mem, [{external_alloc, Sz} | Rest]) -> @@ -4255,7 +4313,7 @@ get_alloc_info(Type, AList) when erlang:is_list(AList) -> Ref = erlang:make_ref(), erlang:system_info({Type, Ref, AList}), receive_allocator(Ref, - erlang:system_info(schedulers), + erts_internal:no_aux_work_threads()-1, mk_res_list(AList)). mk_res_list([]) -> diff --git a/erts/preloaded/src/erts_internal.erl b/erts/preloaded/src/erts_internal.erl index 9925dbd3f9..866b97a094 100644 --- a/erts/preloaded/src/erts_internal.erl +++ b/erts/preloaded/src/erts_internal.erl @@ -113,6 +113,8 @@ -export([prepare_loading/2, beamfile_chunk/2, beamfile_module_md5/1]). +-export([no_aux_work_threads/0]). + %% %% Await result of send to port %% @@ -951,3 +953,8 @@ beamfile_chunk(_Bin, _Chunk) -> -spec beamfile_module_md5(binary()) -> binary() | undefined. beamfile_module_md5(_Bin) -> erlang:nif_error(undefined). + +-spec no_aux_work_threads() -> pos_integer(). + +no_aux_work_threads() -> + erlang:nif_error(undefined). diff --git a/erts/preloaded/src/prim_socket.erl b/erts/preloaded/src/prim_socket.erl index bb6ac414d5..ec92aeb4bf 100644 --- a/erts/preloaded/src/prim_socket.erl +++ b/erts/preloaded/src/prim_socket.erl @@ -43,6 +43,7 @@ setopt/3, setopt_native/3, getopt/2, getopt_native/3, sockname/1, peername/1, + ioctl/2, ioctl/3, ioctl/4, cancel/3 ]). @@ -150,10 +151,13 @@ on_load(Extra) when is_map(Extra) -> init() -> PT = - put_supports_table( - protocols, fun (Protocols) -> protocols_table(Protocols) end), - _ = put_supports_table( - options, fun (Options) -> options_table(Options, PT) end), + put_supports_table(protocols, + fun (Protocols) -> protocols_table(Protocols) end), + _ = put_supports_table(options, + fun (Options) -> options_table(Options, PT) end), + _ = put_supports_table(ioctl_requests, + fun (Requests) -> Requests end), + _ = put_supports_table(ioctl_flags, fun (Flags) -> Flags end), _ = put_supports_table(msg_flags, fun (Flags) -> Flags end), ok. @@ -260,6 +264,13 @@ supports(protocols) -> (Num, _Names, Acc) when is_integer(Num) -> Acc end, [], p_get(protocols)); +supports(ioctl_requests) -> + maps:fold( + fun (Name, _Num, Acc) when is_atom(Name) -> + [{Name, true} | Acc]; + (Num, _Names, Acc) when is_integer(Num) -> + Acc + end, [], p_get(ioctl_requests)); supports(options) -> maps:fold( fun ({_Level,_Opt} = Option, Value, Acc) -> @@ -270,6 +281,11 @@ supports(msg_flags) -> fun (Name, Num, Acc) -> [{Name, Num =/= 0} | Acc] end, [], p_get(msg_flags)); +supports(ioctl_flags) -> + maps:fold( + fun (Name, Num, Acc) -> + [{Name, Num =/= 0} | Acc] + end, [], p_get(ioctl_flags)); supports(Key) -> nif_supports(Key). @@ -305,6 +321,12 @@ is_supported_option(_Option, undefined) -> is_supported(Key1) -> get_is_supported(Key1, nif_supports()). +is_supported(ioctl_requests = Tab, Name) when is_atom(Name) -> + p_get_is_supported(Tab, Name, fun (_) -> true end); +is_supported(ioctl_requests, _Name) -> + false; +is_supported(ioctl_flags = Tab, Flag) -> + p_get_is_supported(Tab, Flag, fun (Value) -> Value =/= 0 end); is_supported(protocols = Tab, Name) when is_atom(Name) -> p_get_is_supported(Tab, Name, fun (_) -> true end); is_supported(protocols, _Name) -> @@ -770,7 +792,7 @@ getopt_result(Error, _Option) -> getopt_native(SockRef, Option, ValueSpec) -> NativeValue = 1, - case enc_sockopt(Option, NativeValue) of + try enc_sockopt(Option, NativeValue) of undefined -> {error, {invalid, {socket_option, Option}}}; invalid -> @@ -784,6 +806,8 @@ getopt_native(SockRef, Option, ValueSpec) -> end; Result -> Result end + catch throw : Reason -> + {error, Reason} end. %% ---------------------------------- @@ -794,15 +818,85 @@ sockname(Ref) -> peername(Ref) -> nif_peername(Ref). + +%% ---------------------------------- + +ioctl(SRef, GReq) -> + case enc_ioctl_request(GReq) of + undefined -> + {error, {invalid, {ioctl_request, GReq}}}; + invalid -> + {error, {invalid, {ioctl_request, GReq}}}; + GReqNUM -> + nif_ioctl(SRef, GReqNUM) + end. + +ioctl(SRef, GReq, Arg) -> + case enc_ioctl_request(GReq) of + undefined -> + {error, {invalid, {ioctl_request, GReq}}}; + invalid -> + {error, {invalid, {ioctl_request, GReq}}}; + GReqNUM -> + nif_ioctl(SRef, GReqNUM, Arg) + end. + + +ioctl(SRef, SReq, Arg1, Arg2) -> + case enc_ioctl_request(SReq) of + undefined -> + {error, {invalid, {ioctl_request, SReq}}}; + invalid -> + {error, {invalid, {ioctl_request, SReq}}}; + SReqNUM when (SReq =:= sifflags) -> + nif_ioctl(SRef, SReqNUM, Arg1, enc_ioctl_flags(Arg2)); + SReqNUM -> + nif_ioctl(SRef, SReqNUM, Arg1, Arg2) + end. + + %% ---------------------------------- cancel(SRef, Op, Ref) -> nif_cancel(SRef, Op, Ref). + %% =========================================================================== %% Encode / decode %% +enc_ioctl_request(GReq) when is_integer(GReq) -> + GReq; +enc_ioctl_request(GReq) -> + case p_get(ioctl_requests) of + #{GReq := GReqNUM} -> + GReqNUM; + #{} -> + invalid + end. + +%% Flags: The flags that shall be set or/and reset +%% #{foo := boolean()} +enc_ioctl_flags(Flags) -> + enc_ioctl_flags(Flags, p_get(ioctl_flags)). + +enc_ioctl_flags(Flags, Table) -> + F = fun(Flag, SetOrReset, FlagMap) when is_boolean(SetOrReset) -> + case Table of + #{Flag := FlagValue} -> + FlagMap#{FlagValue => SetOrReset}; + #{} -> + invalid_ioctl_flag(Flag) + end; + (Flag, BadSetOrReset, _) -> + invalid_ioctl_flag({Flag, BadSetOrReset}) + end, + try maps:fold(F, #{}, Flags) + catch throw : Reason -> + {error, Reason} + end. + + %% These clauses should be deprecated enc_protocol({raw, ProtoNum}) when is_integer(ProtoNum) -> ProtoNum; @@ -996,7 +1090,27 @@ enc_sockopt({Level,Opt} = Option, _NativeValue) end; enc_sockopt(Option, _NativeValue) -> %% Neater than a function clause - erlang:error({invalid, {socket_option, Option}}). + invalid_socket_option(Option). + + +%% =========================================================================== + +-spec invalid_socket_option(Opt :: term()) -> no_return(). + +invalid_socket_option(Opt) -> + invalid({socket_option, Opt}). + +-spec invalid_ioctl_flag(Flag :: term()) -> no_return(). + +invalid_ioctl_flag(Flag) -> + invalid({ioctl_flag, Flag}). + +-spec invalid(What :: term()) -> no_return(). + +invalid(What) -> + throw({invalid, What}). + + %% =========================================================================== %% Persistent term functions @@ -1009,6 +1123,7 @@ p_put(Name, Value) -> p_get(Name) -> persistent_term:get({?MODULE, Name}). + %% =========================================================================== %% NIF functions %% @@ -1060,6 +1175,11 @@ nif_getopt(_SockRef, _Lev, _Opt, _ValSpec) -> erlang:nif_error(undef). nif_sockname(_SockRef) -> erlang:nif_error(undef). nif_peername(_SockRef) -> erlang:nif_error(undef). +nif_ioctl(_SockRef, _GReq) -> erlang:nif_error(undef). +nif_ioctl(_SockRef, _GReq, _Arg) -> erlang:nif_error(undef). +nif_ioctl(_SockRef, _SReq, _Arg1, _Arg2) -> erlang:nif_error(undef). + nif_cancel(_SockRef, _Op, _SelectRef) -> erlang:nif_error(undef). + %% =========================================================================== diff --git a/erts/test/upgrade_SUITE.erl b/erts/test/upgrade_SUITE.erl index 3c1d220b47..72383725cf 100644 --- a/erts/test/upgrade_SUITE.erl +++ b/erts/test/upgrade_SUITE.erl @@ -98,12 +98,18 @@ ancient_major(Config) -> %% If this is a patched version of major release X, then this test %% performs an upgrade from major release X to the current release. minor(Config) -> + put(verbose, true), + p("minor -> get current major release"), CurrentMajor = erlang:system_info(otp_release), Current = CurrentMajor++"_patched", upgrade_test(CurrentMajor,Current,Config). %%%----------------------------------------------------------------- upgrade_test(FromVsn,ToVsn,Config) -> + p("upgrade_test -> entry with" + "~n FromVsn: ~p" + "~n ToVsn: ~p" + "~n Config: ~p", [FromVsn, ToVsn, Config]), OldRel = case test_server:is_release_available(FromVsn) of true -> @@ -121,6 +127,8 @@ upgrade_test(FromVsn,ToVsn,Config) -> end end end, + p("upgrade_test -> old release: " + "~n ~p", [OldRel]), case OldRel of false -> %% Note that priv_dir here is per test case! @@ -131,15 +139,23 @@ upgrade_test(FromVsn,ToVsn,Config) -> end. upgrade_test1(FromVsn,ToVsn,Config) -> + p("upgrade_test1 -> get create-dir"), CreateDir = proplists:get_value(create_dir,Config), + p("upgrade_test1 -> get install-dir"), InstallDir = proplists:get_value(install_dir,Config), FromRelName = "otp-"++FromVsn, ToRelName = "otp-"++ToVsn, {FromRel,FromApps} = target_system(FromRelName, FromVsn, CreateDir, InstallDir,Config), + p("upgrade_test1 -> target system:" + "~n FromRel: ~p" + "~n FromApps: ~p", [FromRel, FromApps]), {ToRel,ToApps} = upgrade_system(FromVsn, FromRel, ToRelName, ToVsn, CreateDir, InstallDir), + p("upgrade_test1 -> upgrade system:" + "~n ToRel: ~p" + "~n ToApps: ~p", [ToRel, ToApps]), do_upgrade(FromVsn, FromApps, ToRel, ToApps, InstallDir). %%%----------------------------------------------------------------- @@ -149,36 +165,45 @@ upgrade_test1(FromVsn,ToVsn,Config) -> %%% - use an own 'start' script %%% - chmod 'start' and 'start_erl' target_system(RelName0,RelVsn,CreateDir,InstallDir,Config) -> + p("target_system -> start worker node"), {ok,Node} = test_server:start_node(list_to_atom(RelName0),peer, [{erl,[proplists:get_value(old_rel,Config)]}]), + p("target_system -> create relfile"), {RelName,Apps,ErtsVsn} = create_relfile(Node,CreateDir,RelName0,RelVsn), %% Create .script and .boot + p("target_system -> create .script and .boot"), ok = rpc:call(Node,systools,make_script,[RelName]), %% Create base tar file - i.e. erts and all apps + p("target_system -> create base tar file"), ok = rpc:call(Node,systools,make_tar, [RelName,[{erts,rpc:call(Node,code,root_dir,[])}]]), %% Unpack the tar to complete the installation + p("target_system -> unpack to complete installation"), erl_tar:extract(RelName ++ ".tar.gz", [{cwd, InstallDir}, compressed]), %% Add bin and log dirs + p("target_system -> add bin and log dirs"), BinDir = filename:join([InstallDir, "bin"]), file:make_dir(BinDir), file:make_dir(filename:join(InstallDir,"log")), %% Delete start scripts - they will be added later + p("target_system -> delete start scripts"), ErtsBinDir = filename:join([InstallDir, "erts-" ++ ErtsVsn, "bin"]), file:delete(filename:join([ErtsBinDir, "erl"])), file:delete(filename:join([ErtsBinDir, "start"])), file:delete(filename:join([ErtsBinDir, "start_erl"])), %% Copy .boot to bin/start.boot + p("target_system -> copy .boot and bin/start.boot"), copy_file(RelName++".boot",filename:join([BinDir, "start.boot"])), %% Copy scripts from erts-xxx/bin to bin + p("target_system -> copy scripts from erts-xxx/bin to bin"), copy_file(filename:join([ErtsBinDir, "epmd"]), filename:join([BinDir, "epmd"]), [preserve]), copy_file(filename:join([ErtsBinDir, "run_erl"]), @@ -187,12 +212,14 @@ target_system(RelName0,RelVsn,CreateDir,InstallDir,Config) -> filename:join([BinDir, "to_erl"]), [preserve]), %% create start_erl.data and sys.config + p("target_system -> create start_erl.data and sys.config"), StartErlData = filename:join([InstallDir, "releases", "start_erl.data"]), write_file(StartErlData, io_lib:fwrite("~s ~s~n", [ErtsVsn, RelVsn])), SysConfig = filename:join([InstallDir, "releases", RelVsn, "sys.config"]), write_file(SysConfig, "[]."), %% Insert 'start' script from data_dir - modified to add sname and heart + p("target_system -> insert 'start' script from data-dir"), copy_file(filename:join(proplists:get_value(data_dir,Config),"start.src"), filename:join(ErtsBinDir,"start.src")), ok = file:change_mode(filename:join(ErtsBinDir,"start.src"),8#0755), @@ -201,23 +228,29 @@ target_system(RelName0,RelVsn,CreateDir,InstallDir,Config) -> %% (this has been fixed in OTP 17 - is is now installed with %% $INSTALL_SCRIPT instead of $INSTALL_DATA and should therefore %% be executable from the start) + p("target_system -> make start_erl executable"), ok = file:change_mode(filename:join(ErtsBinDir,"start_erl.src"),8#0755), %% Substitute variables in erl.src, start.src and start_erl.src %% (.src found in erts-xxx/bin - result stored in bin) + p("target_system -> substitute variables"), subst_src_scripts(["erl", "start", "start_erl"], ErtsBinDir, BinDir, [{"FINAL_ROOTDIR", InstallDir}, {"EMU", "beam"}], [preserve]), %% Create RELEASES + p("target_system -> create releases"), RelFile = filename:join([InstallDir, "releases", filename:basename(RelName) ++ ".rel"]), release_handler:create_RELEASES(InstallDir, RelFile), + p("target_system -> stop worker node"), true = test_server:stop_node(Node), + p("target_system -> done"), {RelName,Apps}. + %%%----------------------------------------------------------------- %%% Create a release containing the current (the test node) OTP %%% release, including relup to allow upgrade from an earlier OTP @@ -283,36 +316,56 @@ fix_relup_inets_ftp(Dir) -> %%% Start a new node running the release from target_system/5 %%% above. Then upgrade to the system from upgrade_system/5. do_upgrade(FromVsn,FromApps,ToRel,ToApps,InstallDir) -> + + p("do_upgrade -> entry with" + "~n FromVsn: ~p" + "~n FromApps: ~p" + "~n ToRel: ~p" + "~n ToApps: ~p" + "~n InstallDir: ~p", + [FromVsn, FromApps, ToRel, ToApps, InstallDir]), + Start = filename:join([InstallDir,bin,start]), + p("do_upgrade -> start (worker from-) node"), {ok,Node} = start_node(Start,permanent,FromVsn,FromApps), + p("do_upgrade -> verify (from-) release (permanent)"), [{"OTP upgrade test",FromVsn,_,permanent}] = rpc:call(Node,release_handler,which_releases,[]), ToRelName = filename:basename(ToRel), + p("do_upgrade -> copy (to-) release"), copy_file(ToRel++".tar.gz", filename:join([InstallDir,releases,ToRelName++".tar.gz"])), + p("do_upgrade -> unpack (to-) release"), {ok,ToVsn} = rpc:call(Node,release_handler,unpack_release,[ToRelName]), + p("do_upgrade -> verify release(s) (unpacked,permanent)"), [{"OTP upgrade test",ToVsn,_,unpacked}, {"OTP upgrade test",FromVsn,_,permanent}] = rpc:call(Node,release_handler,which_releases,[]), + p("do_upgrade -> install (to-) release"), case rpc:call(Node,release_handler,install_release,[ToVsn]) of {ok,FromVsn,_} -> ok; {continue_after_restart,FromVsn,_} -> wait_node_up(current,ToVsn,ToApps) end, + p("do_upgrade -> verify release(s) (current,permanent)"), [{"OTP upgrade test",ToVsn,_,current}, {"OTP upgrade test",FromVsn,_,permanent}] = rpc:call(Node,release_handler,which_releases,[]), + p("do_upgrade -> make (to-) permanent"), ok = rpc:call(Node,release_handler,make_permanent,[ToVsn]), + p("do_upgrade -> verify release(s) (permanent,old)"), [{"OTP upgrade test",ToVsn,_,permanent}, {"OTP upgrade test",FromVsn,_,old}] = rpc:call(Node,release_handler,which_releases,[]), + p("do_upgrade -> stop (worker) node"), erlang:monitor_node(Node,true), _ = rpc:call(Node,init,stop,[]), receive {nodedown,Node} -> ok end, + p("do_upgrade -> done"), ok. %%%----------------------------------------------------------------- @@ -463,24 +516,46 @@ start_node(Start,ExpStatus,ExpVsn,ExpApps) -> wait_node_up(ExpStatus,ExpVsn,ExpApps0) -> ExpApps = [{A,V} || {A,V,_T} <- ExpApps0], - Node = node_name(?upgr_sname), + Node = node_name(?upgr_sname), + erlang:monitor_node(Node, true), wait_node_up(Node,ExpStatus,ExpVsn,lists:keysort(1,ExpApps),60). wait_node_up(Node,ExpStatus,ExpVsn,ExpApps,0) -> + p("wait_node_up -> fail"), + erlang:monitor_node(Node, false), ct:fail({app_check_failed,ExpVsn,ExpApps, - rpc:call(Node,release_handler,which_releases,[ExpStatus]), - rpc:call(Node,application,which_applications,[])}); + rpc:call(Node, release_handler, which_releases, [ExpStatus]), + rpc:call(Node, application, which_applications, [])}); wait_node_up(Node,ExpStatus,ExpVsn,ExpApps,N) -> - timer:sleep(2000), - case {rpc:call(Node,release_handler,which_releases,[ExpStatus]), - rpc:call(Node, application, which_applications, [])} of - {[{_,ExpVsn,_,_}],Apps} when is_list(Apps) -> - case [{A,V} || {A,_,V} <- lists:keysort(1,Apps)] of - ExpApps -> {ok,Node}; - _ -> wait_node_up(Node,ExpStatus,ExpVsn,ExpApps,N-1) - end; - _ -> - wait_node_up(Node,ExpStatus,ExpVsn,ExpApps,N-1) + receive + {nodedown, Node} -> + p("wait_node_up -> [~w] got unexpected (~p) nodedown", [N, Node]), + ct:fail({app_check_failed, ExpVsn, ExpApps, nodedown, Node, N}) + after 2000 -> + p("wait_node_up -> [~w] get release vsn and apps", [N]), + case {rpc:call(Node, release_handler, which_releases, [ExpStatus]), + rpc:call(Node, application, which_applications, [])} of + {[{_,ExpVsn,_,_}],Apps} when is_list(Apps) -> + p("wait_node_up -> [~w] expected release vsn", [N]), + case [{A,V} || {A,_,V} <- lists:keysort(1,Apps)] of + ExpApps -> + p("wait_node_up -> [~w] expected apps", [N]), + erlang:monitor_node(Node, false), + {ok, Node}; + UnexpApps -> + p("wait_node_up -> [~w] still wrong apps:" + "~n ~p", [N, UnexpApps]), + wait_node_up(Node,ExpStatus,ExpVsn,ExpApps,N-1) + end; + {[{_,Vsn,_,_}],_} -> + p("wait_node_up -> [~w] still wrong release vsn:" + "~n ~p", [N, Vsn]), + wait_node_up(Node,ExpStatus,ExpVsn,ExpApps,N-1); + X -> + p("wait_node_up -> [~w] unexpected results:" + "~n ~p", [N, X]), + wait_node_up(Node,ExpStatus,ExpVsn,ExpApps,N-1) + end end. node_name(Sname) -> @@ -499,3 +574,22 @@ rm_rf(Dir) -> _ -> ok end. + + +%%%----------------------------------------------------------------- +%%% + +p(F) -> + p(F, []). + +p(F, A) -> + p(get(verbose), F, A). + +p(true, F, A) -> + print(F, A); +p(_, _, _) -> + ok. + +print(F, A) -> + ct:pal(F ++ "~n", A). + diff --git a/erts/vsn.mk b/erts/vsn.mk index a33a408310..4651777fc4 100644 --- a/erts/vsn.mk +++ b/erts/vsn.mk @@ -18,7 +18,7 @@ # %CopyrightEnd% # -VSN = 12.1.2 +VSN = 12.1.5 # Port number 4365 in 4.2 # Port number 4366 in 4.3 diff --git a/lib/common_test/priv/ct_default.css b/lib/common_test/priv/ct_default.css index ff48b4fdc0..df4fe34f28 100644 --- a/lib/common_test/priv/ct_default.css +++ b/lib/common_test/priv/ct_default.css @@ -207,3 +207,8 @@ th + td { color: #fff; background-color: #809FFF; } + +a.link-to-entry { + float: right; + text-decoration: none; +} diff --git a/lib/common_test/src/ct_gen_conn.erl b/lib/common_test/src/ct_gen_conn.erl index 47aa5966bb..6e0cc76582 100644 --- a/lib/common_test/src/ct_gen_conn.erl +++ b/lib/common_test/src/ct_gen_conn.erl @@ -83,6 +83,7 @@ | {use_existing_connection, boolean()} | {reconnect, boolean()} | {forward_messages, boolean()} + | {old, boolean()} ; (Name, Address, InitData, CallbackMod) -> {ok, handle()} | {error, Reason :: term()} when Name :: ct:target_name(), diff --git a/lib/common_test/src/ct_logs.erl b/lib/common_test/src/ct_logs.erl index 93c15c490c..9e4a5a097e 100644 --- a/lib/common_test/src/ct_logs.erl +++ b/lib/common_test/src/ct_logs.erl @@ -655,7 +655,8 @@ log_timestamp({MS,S,US}) -> tc_groupleaders, stylesheet, async_print_jobs, - tc_esc_chars}). + tc_esc_chars, + log_index}). logger(Parent, Mode, Verbosity) -> register(?MODULE,self()), @@ -786,7 +787,8 @@ logger(Parent, Mode, Verbosity) -> ct_log_fd=CtLogFd, tc_groupleaders=[], async_print_jobs=[], - tc_esc_chars=TcEscChars}). + tc_esc_chars=TcEscChars, + log_index=1}). copy_priv_files([SrcF | SrcFs], [DestF | DestFs]) -> case file:copy(SrcF, DestF) of @@ -925,15 +927,19 @@ logger_loop(State) -> end. create_io_fun(FromPid, CtLogFd, EscChars) -> + create_io_fun(FromPid, CtLogFd, EscChars, undefined). + +create_io_fun(FromPid, CtLogFd, EscChars, LogIndex) -> %% we have to build one io-list of all strings %% before printing, or other io printouts (made in %% parallel) may get printed between this header %% and footer fun(FormatData, IoList) -> - {Escapable,Str,Args} = + {Escapable,AddAnchor,Str,Args} = case FormatData of - {_HdOrFt,S,A} -> {false,S,A}; - {S,A} -> {true,S,A} + {hd,S,A} -> {false,true,S,A}; + {_ft,S,A} -> {false,false,S,A}; + {S,A} -> {true,false,S,A} end, try io_lib:format(lists:flatten(Str), Args) of IoStr when Escapable, EscChars, IoList == [] -> @@ -941,9 +947,9 @@ create_io_fun(FromPid, CtLogFd, EscChars) -> IoStr when Escapable, EscChars -> [IoList,"\n",escape_chars(IoStr)]; IoStr when IoList == [] -> - IoStr; + IoStr++[anchor_link(LogIndex) || AddAnchor]; IoStr -> - [IoList,"\n",IoStr] + [IoList,"\n",IoStr]++[anchor_link(LogIndex) || AddAnchor] catch _:_Reason -> io:format(CtLogFd, "Logging fails! Str: ~tp, Args: ~tp~n", @@ -954,6 +960,13 @@ create_io_fun(FromPid, CtLogFd, EscChars) -> end end. +anchor_link(undefined) -> + []; +anchor_link(LogIndex) -> + IdLink = ["e-", integer_to_list(LogIndex)], + ["<a id=", IdLink, " class=\"link-to-entry\" ", + "href=\"#", IdLink, "\">🔗</a>"]. + escape_chars([Bin | Io]) when is_binary(Bin) -> [Bin | escape_chars(Io)]; escape_chars([List | Io]) when is_list(List) -> @@ -971,12 +984,13 @@ escape_chars([]) -> escape_chars(Bin) -> Bin. -print_to_log(sync, FromPid, Category, TCGL, Content, EscChars, State) -> +print_to_log(sync, FromPid, Category, TCGL, Content, EscChars, + #logger_state{log_index=LogIndex}=State) -> %% in some situations (exceptions), the printout is made from the %% test server IO process and there's no valid group leader to send to CtLogFd = State#logger_state.ct_log_fd, if FromPid /= TCGL -> - IoFun = create_io_fun(FromPid, CtLogFd, EscChars), + IoFun = create_io_fun(FromPid, CtLogFd, EscChars, LogIndex), IoList = lists:foldl(IoFun, [], Content), try tc_io_format(TCGL, "~ts", [IoList]) of ok -> ok @@ -984,19 +998,20 @@ print_to_log(sync, FromPid, Category, TCGL, Content, EscChars, State) -> _:_ -> io:format(TCGL,"~ts", [IoList]) end; - true -> + true -> unexpected_io(FromPid, Category, ?MAX_IMPORTANCE, Content, CtLogFd, EscChars) end, - State; + State#logger_state{log_index=LogIndex+1}; -print_to_log(async, FromPid, Category, TCGL, Content, EscChars, State) -> +print_to_log(async, FromPid, Category, TCGL, Content, EscChars, + #logger_state{log_index=LogIndex}=State) -> %% in some situations (exceptions), the printout is made from the %% test server IO process and there's no valid group leader to send to CtLogFd = State#logger_state.ct_log_fd, Printer = if FromPid /= TCGL -> - IoFun = create_io_fun(FromPid, CtLogFd, EscChars), + IoFun = create_io_fun(FromPid, CtLogFd, EscChars, LogIndex), fun() -> ct_util:mark_process(), test_server:permit_io(TCGL, self()), @@ -1035,12 +1050,13 @@ print_to_log(async, FromPid, Category, TCGL, Content, EscChars, State) -> Content, CtLogFd, EscChars) end end, - case State#logger_state.async_print_jobs of + State1 = State#logger_state{log_index = LogIndex+1}, + case State1#logger_state.async_print_jobs of [] -> {_Pid,Ref} = spawn_monitor(Printer), - State#logger_state{async_print_jobs = [Ref]}; + State1#logger_state{async_print_jobs = [Ref]}; Queue -> - State#logger_state{async_print_jobs = [Printer|Queue]} + State1#logger_state{async_print_jobs = [Printer|Queue]} end. print_next(PrintFun) -> diff --git a/lib/compiler/src/sys_core_fold.erl b/lib/compiler/src/sys_core_fold.erl index 01e9987196..5cc7529469 100644 --- a/lib/compiler/src/sys_core_fold.erl +++ b/lib/compiler/src/sys_core_fold.erl @@ -897,8 +897,9 @@ fold_lit_args(Call, Module, Name, Args0) -> end catch error:Reason -> - %% Evaluation of the function failed. Warn and replace - %% the call with a call to erlang:error/1. + %% Evaluation of the function failed. Warn but keep + %% the call to ensure that extended error information + %% will be available at runtime. eval_failure(Call, Reason) end. @@ -959,15 +960,12 @@ eval_append(Call, X, Y) -> Call#c_call{args=[X,Y]}. %Rebuild call arguments. %% eval_failure(Call, Reason) -> Core. -%% Warn for a call that will fail and replace the call with -%% a call to erlang:error(Reason). +%% Warn for a call that will fail but keep the call. %% eval_failure(Call, Reason) -> Classified = classify_call(Call), add_warning(Call, {failed,{eval_failure,Classified,Reason}}), - Call#c_call{module=#c_literal{val=erlang}, - name=#c_literal{val=error}, - args=[#c_literal{val=Reason}]}. + Call. %% simplify_apply(Call0, Mod, Func, Args) -> Call %% Simplify an apply/3 to a call if the number of arguments diff --git a/lib/compiler/src/v3_core.erl b/lib/compiler/src/v3_core.erl index ffdcea18a2..81d1d59680 100644 --- a/lib/compiler/src/v3_core.erl +++ b/lib/compiler/src/v3_core.erl @@ -1932,11 +1932,12 @@ map_sort_key(Key, KeyMap) -> %% pat_bin([BinElement], State) -> [BinSeg]. pat_bin(Ps0, St) -> - Ps = pat_bin_expand_strings(Ps0), + Ps = pat_bin_expand_strings(Ps0, St), pat_segments(Ps, St). -pat_bin_expand_strings(Es0) -> - foldr(fun ({bin_element,Line,{string,_,[_|_]=S},default,default}, Es1) -> +pat_bin_expand_strings(Es0, #core{dialyzer=Dialyzer}) -> + foldr(fun ({bin_element,Line,{string,_,[_|_]=S},default,default}, Es1) + when not Dialyzer -> bin_expand_string(S, Line, 0, 0, Es1); ({bin_element,Line,{string,_,S},Sz,Ts}, Es1) -> foldr( diff --git a/lib/compiler/test/bs_construct_SUITE.erl b/lib/compiler/test/bs_construct_SUITE.erl index a5f59e8bc3..b314964deb 100644 --- a/lib/compiler/test/bs_construct_SUITE.erl +++ b/lib/compiler/test/bs_construct_SUITE.erl @@ -140,10 +140,16 @@ l(I_13, I_big1, I_16, Bin) -> [129]), ?T(<<1:3,"string",9:5>>, [46,110,142,77,45,204,233]), + + %% Test the native flag. ?T(<<37.98:64/native-float>>, native_3798()), ?T(<<32978297842987249827298387697777669766334937:128/native-integer>>, native_bignum()), + ?T(<<$э/native-utf16,$т/native-utf16,$о/native-utf16," спутник"/native-utf16>>, + native_utf16()), + ?T(<<$в/native-utf32,"ода"/native-utf32>>, + native_utf32()), ?T(<<Bin/binary>>, [165,90,195]), @@ -189,6 +195,18 @@ native_bignum() -> <<1,0>> -> [217,73,11,176,128,109,231,39,101,170,213,1,177,18,205,129] end. +native_utf16() -> + case <<1:16/native>> of + <<0,1>> -> [4,77,4,66,4,62,0,32,4,65,4,63,4,67,4,66,4,61,4,56,4,58]; + <<1,0>> -> [77,4,66,4,62,4,32,0,65,4,63,4,67,4,66,4,61,4,56,4,58,4] + end. + +native_utf32() -> + case <<1:16/native>> of + <<0,1>> -> [0,0,4,50,0,0,4,62,0,0,4,52,0,0,4,48]; + <<1,0>> -> [50,4,0,0,62,4,0,0,52,4,0,0,48,4,0,0] + end. + evaluate(Str, Vars) -> {ok,Tokens,_} = erl_scan:string(Str ++ " . "), @@ -210,13 +228,13 @@ eval_list([{C_bin, Str, Bytes} | Rest], Vars) -> end. one_test({C_bin, E_bin, Str, Bytes}) when is_list(Bytes) -> - io:format(" ~s, ~p~n", [Str, Bytes]), + io:format(" ~ts, ~p~n", [Str, Bytes]), Bin = list_to_bitstring(Bytes), if C_bin == Bin -> ok; true -> - io:format("ERROR: Compiled: ~p. Expected ~p. Got ~p.~n", + io:format("ERROR: Compiled: ~p.~nExpected ~p.~nGot ~p.~n", [Str, Bytes, bitstring_to_list(C_bin)]), ct:fail(comp) end, @@ -224,12 +242,12 @@ one_test({C_bin, E_bin, Str, Bytes}) when is_list(Bytes) -> E_bin == Bin -> ok; true -> - io:format("ERROR: Interpreted: ~p. Expected ~p. Got ~p.~n", + io:format("ERROR: Interpreted: ~p.~nExpected ~p.~nGot ~p.~n", [Str, Bytes, bitstring_to_list(E_bin)]), ct:fail(comp) end; one_test({C_bin, E_bin, Str, Result}) -> - io:format(" ~s ~p~n", [Str, C_bin]), + io:format(" ~ts ~p~n", [Str, C_bin]), if C_bin == E_bin -> ok; diff --git a/lib/compiler/test/guard_SUITE.erl b/lib/compiler/test/guard_SUITE.erl index 69eb7f26b9..d27061ce03 100644 --- a/lib/compiler/test/guard_SUITE.erl +++ b/lib/compiler/test/guard_SUITE.erl @@ -2251,6 +2251,8 @@ bad_guards(Config) when is_list(Config) -> fc(catch bad_guards_4()), + {0,undefined} = bad_guards_5(id(<<>>), id(undefined)), + ok. %% beam_bool used to produce GC BIF instructions whose @@ -2274,6 +2276,15 @@ bad_guards_3(M, [_]) when is_map(M) andalso M#{a := 0, b => 0}, length(M) -> bad_guards_4() when not (error#{}); {not 0.0} -> freedom. +%% The JIT used to segfault when a guard rem instruction failed +%% with badarith AND a bif had been called just before it. +bad_guards_5(A, B) -> + {byte_size(A), undefined = bad_guards_5_1(B)}. +bad_guards_5_1(A) when is_integer(A rem 255) -> + A rem 255; +bad_guards_5_1(_) -> + undefined. + %% Building maps in a guard in a 'catch' would crash v3_codegen. guard_in_catch(_Config) -> diff --git a/lib/compiler/test/warnings_SUITE.erl b/lib/compiler/test/warnings_SUITE.erl index 82ab077cc9..2a287eae9e 100644 --- a/lib/compiler/test/warnings_SUITE.erl +++ b/lib/compiler/test/warnings_SUITE.erl @@ -244,16 +244,12 @@ guard(Config) when is_list(Config) -> ">>, [nowarn_unused_vars], {warnings, - [{{2,15},sys_core_fold,{nomatch,guard}}, - {{2,15},sys_core_fold,{nomatch,no_clause}}, - {{2,28},sys_core_fold, + [{{2,28},sys_core_fold, {failed,{eval_failure, {erlang,element,2}, badarg}}}, {{4,15},sys_core_fold,{nomatch,guard}}, {{4,15},sys_core_fold,{nomatch,no_clause}}, - {{6,15},sys_core_fold,{nomatch,guard}}, - {{6,15},sys_core_fold,{nomatch,no_clause}}, {{6,26},sys_core_fold, {failed, {eval_failure, @@ -284,19 +280,14 @@ bad_arith(Config) when is_list(Config) -> ">>, [], {warnings, - [{{3,19},sys_core_fold,{nomatch,guard}}, - {{3,21},sys_core_fold, + [{{3,21},sys_core_fold, {failed,{eval_failure, {erlang,'+',2}, badarith}}}, - {{9,19},sys_core_fold, - {ignored,{no_effect,{erlang,is_integer,1}}}}, - {{9,19},sys_core_fold,{nomatch,guard}}, {{9,36},sys_core_fold, {failed,{eval_failure, {erlang,'+',2}, badarith}}}, - {{10,19},sys_core_fold,{nomatch,guard}}, {{10,21},sys_core_fold, {failed,{eval_failure, {erlang,'+',2}, diff --git a/lib/crypto/c_src/Makefile.in b/lib/crypto/c_src/Makefile.in index 92918b5d9f..683e4bdbe8 100644 --- a/lib/crypto/c_src/Makefile.in +++ b/lib/crypto/c_src/Makefile.in @@ -111,7 +111,9 @@ CRYPTO_OBJS = $(OBJDIR)/crypto$(TYPEMARKER).o \ $(OBJDIR)/pkey$(TYPEMARKER).o \ $(OBJDIR)/rand$(TYPEMARKER).o \ $(OBJDIR)/rsa$(TYPEMARKER).o \ - $(OBJDIR)/srp$(TYPEMARKER).o + $(OBJDIR)/srp$(TYPEMARKER).o \ + $(OBJDIR)/pbkdf2_hmac$(TYPEMARKER).o + CALLBACK_OBJS = $(OBJDIR)/crypto_callback$(TYPEMARKER).o CRYPTO_STATIC_OBJS = $(patsubst $(OBJDIR)/%$(TYPEMARKER).o,$(OBJDIR)/%_static$(TYPEMARKER).o,$(CRYPTO_OBJS) $(CALLBACK_OBJS)) diff --git a/lib/crypto/c_src/aead.c b/lib/crypto/c_src/aead.c index 8bbeb5bec4..b305117c64 100644 --- a/lib/crypto/c_src/aead.c +++ b/lib/crypto/c_src/aead.c @@ -92,6 +92,8 @@ ERL_NIF_TERM aead_cipher_nif(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[] {ret = EXCP_BADARG_N(env, 0, "Bad cipher"); goto done;} if (! (cipherp->flags & AEAD_CIPHER) ) {ret = EXCP_BADARG_N(env, 0, "Not aead cipher"); goto done;} + if (CIPHER_FORBIDDEN_IN_FIPS(cipherp)) + {ret = EXCP_NOTSUP_N(env, 0, "Forbidden in FIPS"); goto done;} if ((cipher = cipherp->cipher.p) == NULL) {ret = EXCP_NOTSUP_N(env, 0, "The cipher is not supported in this libcrypto version"); goto done;} diff --git a/lib/crypto/c_src/aes.c b/lib/crypto/c_src/aes.c index e3a7c52a2c..feec52d935 100644 --- a/lib/crypto/c_src/aes.c +++ b/lib/crypto/c_src/aes.c @@ -21,92 +21,6 @@ #include "aes.h" #include "cipher.h" -ERL_NIF_TERM aes_cfb_8_crypt(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]) -{/* (Key, IVec, Data, IsEncrypt) */ - ErlNifBinary key, ivec, text; - AES_KEY aes_key; - unsigned char ivec_clone[16]; /* writable copy */ - int new_ivlen = 0; - ERL_NIF_TERM ret; - unsigned char *outp; - - CHECK_NO_FIPS_MODE(); - - ASSERT(argc == 4); - - if (!enif_inspect_iolist_as_binary(env, argv[0], &key)) - goto bad_arg; - if (key.size != 16 && key.size != 24 && key.size != 32) - goto bad_arg; - if (!enif_inspect_binary(env, argv[1], &ivec)) - goto bad_arg; - if (ivec.size != 16) - goto bad_arg; - if (!enif_inspect_iolist_as_binary(env, argv[2], &text)) - goto bad_arg; - - memcpy(ivec_clone, ivec.data, 16); - - /* NOTE: This function returns 0 on success unlike most OpenSSL functions */ - if (AES_set_encrypt_key(key.data, (int)key.size * 8, &aes_key) != 0) - goto err; - if ((outp = enif_make_new_binary(env, text.size, &ret)) == NULL) - goto err; - AES_cfb8_encrypt((unsigned char *) text.data, - outp, - text.size, &aes_key, ivec_clone, &new_ivlen, - (argv[3] == atom_true)); - CONSUME_REDS(env,text); - return ret; - - bad_arg: - err: - return enif_make_badarg(env); -} - -ERL_NIF_TERM aes_cfb_128_crypt_nif(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]) -{/* (Key, IVec, Data, IsEncrypt) */ - ErlNifBinary key, ivec, text; - AES_KEY aes_key; - unsigned char ivec_clone[16]; /* writable copy */ - int new_ivlen = 0; - ERL_NIF_TERM ret; - unsigned char *outp; - - ASSERT(argc == 4); - - if (!enif_inspect_iolist_as_binary(env, argv[0], &key)) - goto bad_arg; - if (key.size != 16 && key.size != 24 && key.size != 32) - goto bad_arg; - if (!enif_inspect_binary(env, argv[1], &ivec)) - goto bad_arg; - if (ivec.size != 16) - goto bad_arg; - if (!enif_inspect_iolist_as_binary(env, argv[2], &text)) - goto bad_arg; - - memcpy(ivec_clone, ivec.data, 16); - - /* NOTE: This function returns 0 on success unlike most OpenSSL functions */ - if (AES_set_encrypt_key(key.data, (int)key.size * 8, &aes_key) != 0) - goto err; - - if ((outp = enif_make_new_binary(env, text.size, &ret)) == NULL) - goto err; - AES_cfb128_encrypt((unsigned char *) text.data, - outp, - text.size, &aes_key, ivec_clone, &new_ivlen, - (argv[3] == atom_true)); - CONSUME_REDS(env,text); - return ret; - - bad_arg: - err: - return enif_make_badarg(env); -} - - #if !defined(HAVE_EVP_AES_CTR) ERL_NIF_TERM aes_ctr_stream_encrypt_compat(ErlNifEnv* env, const ERL_NIF_TERM state_arg, const ERL_NIF_TERM data_arg) {/* ({Key, IVec, ECount, Num}, Data) */ diff --git a/lib/crypto/c_src/aes.h b/lib/crypto/c_src/aes.h index 28a86d891e..f617a7df8f 100644 --- a/lib/crypto/c_src/aes.h +++ b/lib/crypto/c_src/aes.h @@ -23,9 +23,6 @@ #include "common.h" -ERL_NIF_TERM aes_cfb_8_crypt(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]); -ERL_NIF_TERM aes_cfb_128_crypt_nif(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]); - #if !defined(HAVE_EVP_AES_CTR) ERL_NIF_TERM aes_ctr_stream_encrypt_compat(ErlNifEnv* env, const ERL_NIF_TERM state_arg, const ERL_NIF_TERM data_arg); #endif diff --git a/lib/crypto/c_src/algorithms.c b/lib/crypto/c_src/algorithms.c index 4de595a934..e7087bd461 100644 --- a/lib/crypto/c_src/algorithms.c +++ b/lib/crypto/c_src/algorithms.c @@ -65,10 +65,7 @@ void cleanup_algorithms_types(ErlNifEnv* env) ERL_NIF_TERM hash_algorithms(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]) { unsigned int cnt = -#ifdef FIPS_SUPPORT - FIPS_mode() ? algo_hash_fips_cnt : -#endif - algo_hash_cnt; + FIPS_MODE() ? algo_hash_fips_cnt : algo_hash_cnt; return enif_make_list_from_array(env, algo_hash, cnt); } @@ -129,10 +126,7 @@ void init_hash_types(ErlNifEnv* env) { ERL_NIF_TERM pubkey_algorithms(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]) { unsigned int cnt = -#ifdef FIPS_SUPPORT - FIPS_mode() ? algo_pubkey_fips_cnt : -#endif - algo_pubkey_cnt; + FIPS_MODE() ? algo_pubkey_fips_cnt : algo_pubkey_cnt; return enif_make_list_from_array(env, algo_pubkey, cnt); } @@ -195,14 +189,12 @@ ERL_NIF_TERM mac_algorithms(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]) ERL_NIF_TERM curve_algorithms(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]) { - int fips_mode = 0; - int algo_curve_cnt = 0; - -# ifdef FIPS_SUPPORT - if (FIPS_mode()) fips_mode = 1; -# endif + int fips_mode; + int algo_curve_cnt; + fips_mode = FIPS_MODE(); algo_curve_cnt = get_curve_cnt(env, fips_mode); + return enif_make_list_from_array(env, algo_curve[fips_mode], algo_curve_cnt); } @@ -247,7 +239,7 @@ void init_curve_types(ErlNifEnv* env) { by calling get_curve_cnt */ #ifdef FIPS_SUPPORT - if (FIPS_mode()) { + if (FIPS_MODE()) { // FIPS enabled get_curve_cnt(env, 1); FIPS_mode_set(0); // disable @@ -695,10 +687,7 @@ int valid_curve(int nid) { ERL_NIF_TERM rsa_opts_algorithms(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]) { unsigned int cnt = -#ifdef FIPS_SUPPORT - FIPS_mode() ? algo_rsa_opts_fips_cnt : -#endif - algo_rsa_opts_cnt; + FIPS_MODE() ? algo_rsa_opts_fips_cnt : algo_rsa_opts_cnt; return enif_make_list_from_array(env, algo_rsa_opts, cnt); } diff --git a/lib/crypto/c_src/api_ng.c b/lib/crypto/c_src/api_ng.c index 35404ba1e0..b745974d14 100644 --- a/lib/crypto/c_src/api_ng.c +++ b/lib/crypto/c_src/api_ng.c @@ -200,16 +200,22 @@ static int get_init_args(ErlNifEnv* env, ERL_NIF_TERM *return_term) { int ivec_len; - ErlNifBinary key_bin; ErlNifBinary ivec_bin; ctx_res->ctx = NULL; /* For testing if *ctx should be freed after errors */ #if !defined(HAVE_EVP_AES_CTR) ctx_res->env = NULL; /* For testing if *env should be freed after errors */ #endif + ctx_res->key_bin.data = NULL; + ctx_res->padding = atom_undefined; ctx_res->padded_size = -1; ctx_res->size = 0; + /* Two initializations to make CodeChecker happy: it gets a bit desoriented + by the NIF Exception model */ + ctx_res->encflag = 0; + ctx_res->padding = atom_error; + /* Fetch the options */ if ((*return_term = get_opts(env, argv[opts_arg_num], opts_arg_num, &(ctx_res->encflag), &(ctx_res->padding)) @@ -217,7 +223,7 @@ static int get_init_args(ErlNifEnv* env, goto err; /* Fetch the key */ - if (!enif_inspect_iolist_as_binary(env, argv[key_arg_num], &key_bin)) + if (!enif_inspect_iolist_as_binary(env, argv[key_arg_num], &ctx_res->key_bin)) { *return_term = EXCP_BADARG_N(env, key_arg_num, "Bad key"); goto err; @@ -230,7 +236,7 @@ static int get_init_args(ErlNifEnv* env, goto err; } - if (!(*cipherp = get_cipher_type(argv[cipher_arg_num], key_bin.size))) + if (!(*cipherp = get_cipher_type(argv[cipher_arg_num], ctx_res->key_bin.size))) { if (!get_cipher_type_no_key(argv[cipher_arg_num])) *return_term = EXCP_BADARG_N(env, cipher_arg_num, "Unknown cipher"); @@ -344,7 +350,7 @@ static int get_init_args(ErlNifEnv* env, goto err; } - if (!EVP_CIPHER_CTX_set_key_length(ctx_res->ctx, (int)key_bin.size)) + if (!EVP_CIPHER_CTX_set_key_length(ctx_res->ctx, (int)ctx_res->key_bin.size)) { *return_term = EXCP_ERROR_N(env, key_arg_num, "Can't initialize context, key_length"); goto err; @@ -352,11 +358,11 @@ static int get_init_args(ErlNifEnv* env, #ifdef HAVE_RC2 if (EVP_CIPHER_type((*cipherp)->cipher.p) == NID_rc2_cbc) { - if (key_bin.size > INT_MAX / 8) { + if (ctx_res->key_bin.size > INT_MAX / 8) { *return_term = EXCP_BADARG_N(env, key_arg_num, "To large rc2_cbc key"); goto err; } - if (!EVP_CIPHER_CTX_ctrl(ctx_res->ctx, EVP_CTRL_SET_RC2_KEY_BITS, (int)key_bin.size * 8, NULL)) { + if (!EVP_CIPHER_CTX_ctrl(ctx_res->ctx, EVP_CTRL_SET_RC2_KEY_BITS, (int)ctx_res->key_bin.size * 8, NULL)) { *return_term = EXCP_BADARG_N(env, key_arg_num, "ctrl rc2_cbc key"); goto err; } @@ -365,13 +371,13 @@ static int get_init_args(ErlNifEnv* env, if (argv[ivec_arg_num] == atom_undefined || ivec_len == 0) { - if (!EVP_CipherInit_ex(ctx_res->ctx, NULL, NULL, key_bin.data, NULL, -1)) { + if (!EVP_CipherInit_ex(ctx_res->ctx, NULL, NULL,ctx_res->key_bin.data, NULL, -1)) { *return_term = EXCP_BADARG_N(env, key_arg_num, "Can't initialize key"); goto err; } } else - if (!EVP_CipherInit_ex(ctx_res->ctx, NULL, NULL, key_bin.data, ivec_bin.data, -1)) + if (!EVP_CipherInit_ex(ctx_res->ctx, NULL, NULL, ctx_res->key_bin.data, ivec_bin.data, -1)) { *return_term = EXCP_ERROR(env, "Can't initialize key or iv"); goto err; @@ -727,11 +733,50 @@ ERL_NIF_TERM ng_crypto_update(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[ #endif { ctx_res_copy.ctx = EVP_CIPHER_CTX_new(); + if (! ctx_res->ctx) + { + ret = EXCP_ERROR(env, "Can't allocate context"); + goto err; + } +#if OPENSSL_VERSION_NUMBER < PACKED_OPENSSL_VERSION_PLAIN(3,0,0) if (!EVP_CIPHER_CTX_copy(ctx_res_copy.ctx, ctx_res->ctx)) { ret = EXCP_ERROR(env, "Can't copy ctx_res"); goto err; } +#else + + if (!EVP_CipherInit_ex(ctx_res_copy.ctx, + EVP_CIPHER_CTX_cipher(ctx_res->ctx), + NULL, NULL, NULL, ctx_res->encflag)) + { + ret = EXCP_ERROR(env, "Can't initialize context, step 1"); + goto err; + } + + + if (!EVP_CIPHER_CTX_set_key_length(ctx_res_copy.ctx, + ctx_res->key_bin.size)) + { + ret = EXCP_ERROR(env, "Can't initialize context, key_length"); + goto err; + } + + + if (!EVP_CipherInit_ex(ctx_res_copy.ctx, NULL, NULL, + ctx_res->key_bin.data, + NULL, -1)) + { + ret = EXCP_ERROR(env, "Can't initialize key or iv"); + goto err; + } + + if ((ctx_res->padding == atom_undefined) || + (ctx_res->padding == atom_none) || + (ctx_res->padding == atom_zero) || + (ctx_res->padding == atom_random) ) + EVP_CIPHER_CTX_set_padding(ctx_res_copy.ctx, 0); +#endif } if (!enif_inspect_iolist_as_binary(env, argv[2], &ivec_bin)) @@ -758,14 +803,15 @@ ERL_NIF_TERM ng_crypto_update(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[ ctx_res_copy.state = enif_make_tuple4(env, tuple_argv[0], argv[2], tuple_argv[2], tuple_argv[3]); } } else -#endif - if (!EVP_CipherInit_ex(ctx_res_copy.ctx, NULL, NULL, NULL, ivec_bin.data, -1)) +#endif + if (!EVP_CipherInit_ex(ctx_res_copy.ctx, NULL, NULL, NULL, ivec_bin.data, -1)) { ret = EXCP_ERROR(env, "Can't set iv"); goto err; } get_update_args(env, &ctx_res_copy, argv, 1, &ret); + ctx_res->size = ctx_res_copy.size; } else /* argc != 3, that is, argc = 2 (we don't have an IV in this call) */ diff --git a/lib/crypto/c_src/atoms.c b/lib/crypto/c_src/atoms.c index cae3c5287d..b6a190bf09 100644 --- a/lib/crypto/c_src/atoms.c +++ b/lib/crypto/c_src/atoms.c @@ -44,12 +44,9 @@ ERL_NIF_TERM atom_none; ERL_NIF_TERM atom_notsup; ERL_NIF_TERM atom_badarg; ERL_NIF_TERM atom_digest; -#ifdef FIPS_SUPPORT ERL_NIF_TERM atom_enabled; ERL_NIF_TERM atom_not_enabled; -#else ERL_NIF_TERM atom_not_supported; -#endif ERL_NIF_TERM atom_type; ERL_NIF_TERM atom_size; @@ -230,12 +227,10 @@ int init_atoms(ErlNifEnv *env, const ERL_NIF_TERM fips_mode, const ERL_NIF_TERM atom_aes_ccm = enif_make_atom(env, "aes_ccm"); #endif -#ifdef FIPS_SUPPORT atom_enabled = enif_make_atom(env,"enabled"); atom_not_enabled = enif_make_atom(env,"not_enabled"); -#else atom_not_supported = enif_make_atom(env,"not_supported"); -#endif + atom_rsa = enif_make_atom(env,"rsa"); atom_dss = enif_make_atom(env,"dss"); atom_ecdsa = enif_make_atom(env,"ecdsa"); diff --git a/lib/crypto/c_src/atoms.h b/lib/crypto/c_src/atoms.h index d6bdc43a52..7bedf799b6 100644 --- a/lib/crypto/c_src/atoms.h +++ b/lib/crypto/c_src/atoms.h @@ -48,12 +48,9 @@ extern ERL_NIF_TERM atom_none; extern ERL_NIF_TERM atom_notsup; extern ERL_NIF_TERM atom_badarg; extern ERL_NIF_TERM atom_digest; -#ifdef FIPS_SUPPORT extern ERL_NIF_TERM atom_enabled; extern ERL_NIF_TERM atom_not_enabled; -#else extern ERL_NIF_TERM atom_not_supported; -#endif extern ERL_NIF_TERM atom_type; extern ERL_NIF_TERM atom_size; diff --git a/lib/crypto/c_src/cipher.c b/lib/crypto/c_src/cipher.c index 284ea0ba81..fbda4a688d 100644 --- a/lib/crypto/c_src/cipher.c +++ b/lib/crypto/c_src/cipher.c @@ -20,12 +20,6 @@ #include "cipher.h" -#ifdef HAVE_DES -#define COND_NO_DES_PTR(Ptr) (Ptr) -#else -#define COND_NO_DES_PTR(Ptr) (NULL) -#endif - #define NOT_AEAD {{0,0,0}} static struct cipher_type_t cipher_types[] = @@ -33,21 +27,33 @@ static struct cipher_type_t cipher_types[] = #ifdef HAVE_RC2 {{"rc2_cbc"}, {&EVP_rc2_cbc}, 0, NO_FIPS_CIPHER, NOT_AEAD}, #else - {{"rc2_cbc"}, {NULL}, 0, NO_FIPS_CIPHER, NOT_AEAD}, + {{"rc2_cbc"}, {NULL}, 0, 0}, #endif + #ifdef HAVE_RC4 {{"rc4"}, {&EVP_rc4}, 0, NO_FIPS_CIPHER, NOT_AEAD}, #else {{"rc4"}, {NULL}, 0, NO_FIPS_CIPHER, NOT_AEAD}, #endif - {{"des_cbc"}, {COND_NO_DES_PTR(&EVP_des_cbc)}, 0, NO_FIPS_CIPHER, NOT_AEAD}, - {{"des_cfb"}, {COND_NO_DES_PTR(&EVP_des_cfb8)}, 0, NO_FIPS_CIPHER, NOT_AEAD}, - {{"des_ecb"}, {COND_NO_DES_PTR(&EVP_des_ecb)}, 0, NO_FIPS_CIPHER | ECB_BUG_0_9_8L, NOT_AEAD}, - {{"des_ede3_cbc"}, {COND_NO_DES_PTR(&EVP_des_ede3_cbc)}, 0, 0, NOT_AEAD}, +#ifdef HAVE_DES + {{"des_cbc"}, {&EVP_des_cbc}, 0, NO_FIPS_CIPHER}, + {{"des_cfb"}, {&EVP_des_cfb8}, 0, NO_FIPS_CIPHER}, + {{"des_ecb"}, {&EVP_des_ecb}, 0, NO_FIPS_CIPHER | ECB_BUG_0_9_8L}, +#else + {{"des_cbc"}, {NULL}, 0, 0}, + {{"des_cfb"}, {NULL}, 0, 0}, + {{"des_ecb"}, {NULL}, 0, 0}, +#endif + +#ifdef HAVE_DES_ede3_cbc + {{"des_ede3_cbc"}, {&EVP_des_ede3_cbc}, 0, 0}, +#else + {{"des_ede3_cbc"}, {NULL}, 0, 0}, +#endif -#ifdef HAVE_DES_ede3_cfb_encrypt - {{"des_ede3_cfb"}, {COND_NO_DES_PTR(&EVP_des_ede3_cfb8)}, 0, 0, NOT_AEAD}, +#ifdef HAVE_DES_ede3_cfb + {{"des_ede3_cfb"}, {&EVP_des_ede3_cfb8}, 0, 0}, #else {{"des_ede3_cfb"}, {NULL}, 0, 0, NOT_AEAD}, #endif @@ -103,7 +109,11 @@ static struct cipher_type_t cipher_types[] = {{"chacha20_poly1305"}, {NULL}, 0, NO_FIPS_CIPHER | AEAD_CIPHER, {{0,0,0}}}, #endif -#if defined(HAVE_GCM) +#if defined(HAVE_GCM) && OPENSSL_VERSION_NUMBER >= PACKED_OPENSSL_VERSION_PLAIN(3,0,0) + {{"aes_128_gcm"}, {&EVP_aes_128_gcm}, 16, AEAD_CIPHER|GCM_MODE, {{EVP_CTRL_AEAD_SET_IVLEN,EVP_CTRL_AEAD_GET_TAG,EVP_CTRL_AEAD_SET_TAG}}}, + {{"aes_192_gcm"}, {&EVP_aes_192_gcm}, 24, AEAD_CIPHER|GCM_MODE, {{EVP_CTRL_AEAD_SET_IVLEN,EVP_CTRL_AEAD_GET_TAG,EVP_CTRL_AEAD_SET_TAG}}}, + {{"aes_256_gcm"}, {&EVP_aes_256_gcm}, 32, AEAD_CIPHER|GCM_MODE, {{EVP_CTRL_AEAD_SET_IVLEN,EVP_CTRL_AEAD_GET_TAG,EVP_CTRL_AEAD_SET_TAG}}}, +#elif defined(HAVE_GCM) {{"aes_128_gcm"}, {&EVP_aes_128_gcm}, 16, AEAD_CIPHER|GCM_MODE, {{EVP_CTRL_GCM_SET_IVLEN,EVP_CTRL_GCM_GET_TAG,EVP_CTRL_GCM_SET_TAG}}}, {{"aes_192_gcm"}, {&EVP_aes_192_gcm}, 24, AEAD_CIPHER|GCM_MODE, {{EVP_CTRL_GCM_SET_IVLEN,EVP_CTRL_GCM_GET_TAG,EVP_CTRL_GCM_SET_TAG}}}, {{"aes_256_gcm"}, {&EVP_aes_256_gcm}, 32, AEAD_CIPHER|GCM_MODE, {{EVP_CTRL_GCM_SET_IVLEN,EVP_CTRL_GCM_GET_TAG,EVP_CTRL_GCM_SET_TAG}}}, @@ -113,7 +123,11 @@ static struct cipher_type_t cipher_types[] = {{"aes_256_gcm"}, {NULL}, 32, AEAD_CIPHER|GCM_MODE, {{0,0,0}}}, #endif -#if defined(HAVE_CCM) +#if defined(HAVE_CCM) && OPENSSL_VERSION_NUMBER >= PACKED_OPENSSL_VERSION_PLAIN(3,0,0) + {{"aes_128_ccm"}, {&EVP_aes_128_ccm}, 16, AEAD_CIPHER|CCM_MODE, {{EVP_CTRL_AEAD_SET_IVLEN,EVP_CTRL_AEAD_GET_TAG,EVP_CTRL_AEAD_SET_TAG}}}, + {{"aes_192_ccm"}, {&EVP_aes_192_ccm}, 24, AEAD_CIPHER|CCM_MODE, {{EVP_CTRL_AEAD_SET_IVLEN,EVP_CTRL_AEAD_GET_TAG,EVP_CTRL_AEAD_SET_TAG}}}, + {{"aes_256_ccm"}, {&EVP_aes_256_ccm}, 32, AEAD_CIPHER|CCM_MODE, {{EVP_CTRL_AEAD_SET_IVLEN,EVP_CTRL_AEAD_GET_TAG,EVP_CTRL_AEAD_SET_TAG}}}, +#elif defined(HAVE_CCM) {{"aes_128_ccm"}, {&EVP_aes_128_ccm}, 16, AEAD_CIPHER|CCM_MODE, {{EVP_CTRL_CCM_SET_IVLEN,EVP_CTRL_CCM_GET_TAG,EVP_CTRL_CCM_SET_TAG}}}, {{"aes_192_ccm"}, {&EVP_aes_192_ccm}, 24, AEAD_CIPHER|CCM_MODE, {{EVP_CTRL_CCM_SET_IVLEN,EVP_CTRL_CCM_GET_TAG,EVP_CTRL_CCM_SET_TAG}}}, {{"aes_256_ccm"}, {&EVP_aes_256_ccm}, 32, AEAD_CIPHER|CCM_MODE, {{EVP_CTRL_CCM_SET_IVLEN,EVP_CTRL_CCM_GET_TAG,EVP_CTRL_CCM_SET_TAG}}}, @@ -347,18 +361,5 @@ ERL_NIF_TERM cipher_types_as_list(ErlNifEnv* env) } } - /* Add aliases: */ -#ifdef HAVE_GCM - hd = enif_make_list_cell(env, atom_aes_gcm, hd); -#endif - hd = enif_make_list_cell(env, atom_aes_ecb, hd); - hd = enif_make_list_cell(env, atom_aes_ctr, hd); - hd = enif_make_list_cell(env, atom_aes_cfb8, hd); - hd = enif_make_list_cell(env, atom_aes_cfb128, hd); -#ifdef HAVE_CCM - hd = enif_make_list_cell(env, atom_aes_ccm, hd); -#endif - hd = enif_make_list_cell(env, atom_aes_cbc, hd); - return hd; } diff --git a/lib/crypto/c_src/cipher.h b/lib/crypto/c_src/cipher.h index d4c1bed98d..85d2f4e14c 100644 --- a/lib/crypto/c_src/cipher.h +++ b/lib/crypto/c_src/cipher.h @@ -49,12 +49,9 @@ struct cipher_type_t { #define CCM_MODE 64 #define GCM_MODE 128 - #ifdef FIPS_SUPPORT -/* May have FIPS support, must check dynamically if it is enabled */ -# define CIPHER_FORBIDDEN_IN_FIPS(P) (((P)->flags & NO_FIPS_CIPHER) && FIPS_mode()) +# define CIPHER_FORBIDDEN_IN_FIPS(P) (((P)->flags & NO_FIPS_CIPHER) && FIPS_MODE()) #else -/* No FIPS support since the symbol FIPS_SUPPORT is undefined */ # define CIPHER_FORBIDDEN_IN_FIPS(P) 0 #endif @@ -63,6 +60,7 @@ struct evp_cipher_ctx { EVP_CIPHER_CTX* ctx; int iv_len; ERL_NIF_TERM padding; /* id of the padding to add by get_final_args() */ + ErlNifBinary key_bin; int padded_size; /* Length of the padding that was added */ int encflag; /* 1 if encrypting, 0 if decrypting */ unsigned int size; /* The sum of all sizes of input texts to get_update_args() */ diff --git a/lib/crypto/c_src/crypto.c b/lib/crypto/c_src/crypto.c index 65804bf6fb..3397b767eb 100644 --- a/lib/crypto/c_src/crypto.c +++ b/lib/crypto/c_src/crypto.c @@ -45,6 +45,7 @@ #include "hmac.h" #include "info.h" #include "math.h" +#include "pbkdf2_hmac.h" #include "pkey.h" #include "rand.h" #include "rsa.h" @@ -90,6 +91,7 @@ static ErlNifFunc nif_funcs[] = { {"rand_uniform_nif", 2, rand_uniform_nif, 0}, {"mod_exp_nif", 4, mod_exp_nif, 0}, {"do_exor", 2, do_exor, 0}, + {"pbkdf2_hmac_nif", 5, pbkdf2_hmac_nif, 0}, {"pkey_sign_nif", 5, pkey_sign_nif, 0}, {"pkey_verify_nif", 6, pkey_verify_nif, 0}, {"pkey_crypt_nif", 6, pkey_crypt_nif, 0}, diff --git a/lib/crypto/c_src/digest.c b/lib/crypto/c_src/digest.c index d4b1813e11..125784ac42 100644 --- a/lib/crypto/c_src/digest.c +++ b/lib/crypto/c_src/digest.c @@ -46,9 +46,9 @@ static struct digest_type_t digest_types[] = #endif }, - {{"sha"}, 0, {&EVP_sha1}}, + {{"sha"}, PBKDF2_ELIGIBLE_DIGEST, {&EVP_sha1}}, - {{"sha224"}, 0, + {{"sha224"}, PBKDF2_ELIGIBLE_DIGEST, #ifdef HAVE_SHA224 {&EVP_sha224} #else @@ -56,7 +56,7 @@ static struct digest_type_t digest_types[] = #endif }, - {{"sha256"}, 0, + {{"sha256"}, PBKDF2_ELIGIBLE_DIGEST, #ifdef HAVE_SHA256 {&EVP_sha256} #else @@ -64,7 +64,7 @@ static struct digest_type_t digest_types[] = #endif }, - {{"sha384"}, 0, + {{"sha384"}, PBKDF2_ELIGIBLE_DIGEST, #ifdef HAVE_SHA384 {&EVP_sha384} #else @@ -72,7 +72,7 @@ static struct digest_type_t digest_types[] = #endif }, - {{"sha512"}, 0, + {{"sha512"}, PBKDF2_ELIGIBLE_DIGEST, #ifdef HAVE_SHA512 {&EVP_sha512} #else diff --git a/lib/crypto/c_src/digest.h b/lib/crypto/c_src/digest.h index 76fe14a06a..8d8cb243d1 100644 --- a/lib/crypto/c_src/digest.h +++ b/lib/crypto/c_src/digest.h @@ -37,12 +37,11 @@ struct digest_type_t { /* masks in the flags field if digest_type_t */ #define NO_FIPS_DIGEST 1 +#define PBKDF2_ELIGIBLE_DIGEST 2 #ifdef FIPS_SUPPORT -/* May have FIPS support, must check dynamically if it is enabled */ -# define DIGEST_FORBIDDEN_IN_FIPS(P) (((P)->flags & NO_FIPS_DIGEST) && FIPS_mode()) +# define DIGEST_FORBIDDEN_IN_FIPS(P) (((P)->flags & NO_FIPS_DIGEST) && FIPS_MODE()) #else -/* No FIPS support since the symbol FIPS_SUPPORT is undefined */ # define DIGEST_FORBIDDEN_IN_FIPS(P) 0 #endif diff --git a/lib/crypto/c_src/fips.c b/lib/crypto/c_src/fips.c index 072aade7b9..046169ad8d 100644 --- a/lib/crypto/c_src/fips.c +++ b/lib/crypto/c_src/fips.c @@ -23,7 +23,7 @@ ERL_NIF_TERM info_fips(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]) { #ifdef FIPS_SUPPORT - return FIPS_mode() ? atom_enabled : atom_not_enabled; + return FIPS_MODE() ? atom_enabled : atom_not_enabled; #else return atom_not_supported; #endif diff --git a/lib/crypto/c_src/mac.c b/lib/crypto/c_src/mac.c index 13bd2815b1..747d5c7f39 100644 --- a/lib/crypto/c_src/mac.c +++ b/lib/crypto/c_src/mac.c @@ -85,16 +85,12 @@ static struct mac_type_t mac_types[] = } }; - #ifdef FIPS_SUPPORT -/* May have FIPS support, must check dynamically if it is enabled */ -# define MAC_FORBIDDEN_IN_FIPS(P) (((P)->flags & NO_FIPS_MAC) && FIPS_mode()) +# define MAC_FORBIDDEN_IN_FIPS(P) (((P)->flags & NO_FIPS_MAC) && FIPS_MODE()) #else -/* No FIPS support since the symbol FIPS_SUPPORT is undefined */ # define MAC_FORBIDDEN_IN_FIPS(P) 0 #endif - /*************************** Mandatory prototypes ***************************/ diff --git a/lib/crypto/c_src/openssl_config.h b/lib/crypto/c_src/openssl_config.h index 9ef04fca3c..ac089b6ba8 100644 --- a/lib/crypto/c_src/openssl_config.h +++ b/lib/crypto/c_src/openssl_config.h @@ -49,15 +49,7 @@ #include <openssl/evp.h> #include <openssl/hmac.h> #include <openssl/err.h> - -/* Helper macro to construct a OPENSSL_VERSION_NUMBER. - * See openssl/opensslv.h - */ -#define PACKED_OPENSSL_VERSION(MAJ, MIN, FIX, P) \ - ((((((((MAJ << 8) | MIN) << 8 ) | FIX) << 8) | (P-'a'+1)) << 4) | 0xf) - -#define PACKED_OPENSSL_VERSION_PLAIN(MAJ, MIN, FIX) \ - PACKED_OPENSSL_VERSION(MAJ,MIN,FIX,('a'-1)) +#include "openssl_version.h" /* LibreSSL was cloned from OpenSSL 1.0.1g and claims to be API and BPI compatible @@ -78,21 +70,9 @@ * */ -#ifdef LIBRESSL_VERSION_NUMBER -/* A macro to test on in this file */ -#define HAS_LIBRESSL -#endif - #ifdef HAS_LIBRESSL /* LibreSSL dislikes FIPS */ -# ifdef FIPS_SUPPORT # undef FIPS_SUPPORT -# endif - -/* LibreSSL has never supported the custom mem functions */ -#ifndef HAS_LIBRESSL -# define HAS_CRYPTO_MEM_FUNCTIONS -#endif # if LIBRESSL_VERSION_NUMBER < PACKED_OPENSSL_VERSION_PLAIN(2,7,0) /* LibreSSL wants the 1.0.1 API */ @@ -101,6 +81,19 @@ #endif +/* LibreSSL has never supported the custom mem functions */ +#ifndef HAS_LIBRESSL +/* Since f46401d46f9ed331ff2a09bb6a99376707083c96 this macro can NEVER have been enabled + * because its inside an #ifdef HAS_LIBRESSL + * + * If I enable HAS_CRYPTO_MEM_FUNCTIONS, there are two lab machines that fails: + * SunOS mallor 5.11 illumos-2d990ab13b i86pc i386 i86pc OpenSSL 1.0.2u 20 Dec 2019 + * SunOS fingon 5.11 11.4.0.15.0 i86pc i386 i86pc OpenSSL 1.0.2o 27 Mar 2018 + * + * Therefore I don't want to enable this until further investigated +# define HAS_CRYPTO_MEM_FUNCTIONS + */ +#endif #if OPENSSL_VERSION_NUMBER < PACKED_OPENSSL_VERSION_PLAIN(1,1,0) # define NEED_EVP_COMPATIBILITY_FUNCTIONS @@ -160,9 +153,6 @@ && !defined(OPENSSL_NO_SHA512) && defined(NID_sha512) # define HAVE_SHA512 #endif -#if OPENSSL_VERSION_NUMBER >= PACKED_OPENSSL_VERSION(0,9,7,'e') -# define HAVE_DES_ede3_cfb_encrypt -#endif // SHA3: #if OPENSSL_VERSION_NUMBER >= PACKED_OPENSSL_VERSION_PLAIN(1,1,1) @@ -188,14 +178,24 @@ # define HAVE_BLAKE2 #endif -#ifndef OPENSSL_NO_BF +#if !defined(OPENSSL_NO_BF) \ + && OPENSSL_VERSION_NUMBER < PACKED_OPENSSL_VERSION_PLAIN(3,0,0) # define HAVE_BF #endif -#ifndef OPENSSL_NO_DES +#if !defined(OPENSSL_NO_DES) \ + && OPENSSL_VERSION_NUMBER < PACKED_OPENSSL_VERSION_PLAIN(3,0,0) # define HAVE_DES #endif +#if OPENSSL_VERSION_NUMBER >= PACKED_OPENSSL_VERSION(0,9,7,'e') +# define HAVE_DES_ede3_cfb +#endif + +#if OPENSSL_VERSION_NUMBER >= PACKED_OPENSSL_VERSION(0,9,7,'e') +# define HAVE_DES_ede3_cbc +#endif + #ifndef OPENSSL_NO_DH # define HAVE_DH #endif @@ -204,7 +204,8 @@ # define HAVE_DSA #endif -#ifndef OPENSSL_NO_MD4 +#if !defined(OPENSSL_NO_MD4) \ + && OPENSSL_VERSION_NUMBER < PACKED_OPENSSL_VERSION_PLAIN(3,0,0) # define HAVE_MD4 #endif @@ -212,16 +213,20 @@ # define HAVE_MD5 #endif -#ifndef OPENSSL_NO_RC2 +#if !defined(OPENSSL_NO_RC2) \ + && OPENSSL_VERSION_NUMBER < PACKED_OPENSSL_VERSION_PLAIN(3,0,0) # define HAVE_RC2 #endif -#ifndef OPENSSL_NO_RC4 +#if !defined(OPENSSL_NO_RC4) \ + && OPENSSL_VERSION_NUMBER < PACKED_OPENSSL_VERSION_PLAIN(3,0,0) # define HAVE_RC4 #endif -#ifndef OPENSSL_NO_RMD160 -/* Note RMD160 vs RIPEMD160 */ +#if !defined(OPENSSL_NO_RMD160) && \ + !defined(OPENSSL_NO_RIPEMD160) && \ + !defined(OPENSSL_NO_RIPEMD) && \ + OPENSSL_VERSION_NUMBER < PACKED_OPENSSL_VERSION_PLAIN(3,0,0) # define HAVE_RIPEMD160 #endif @@ -245,6 +250,11 @@ # endif #endif +#if OPENSSL_VERSION_NUMBER >= PACKED_OPENSSL_VERSION(0,9,8,'c') \ + && OPENSSL_VERSION_NUMBER < PACKED_OPENSSL_VERSION_PLAIN(3,0,0) +# define HAVE_AES_IGE +#endif + #if OPENSSL_VERSION_NUMBER >= PACKED_OPENSSL_VERSION_PLAIN(1,0,1) # define HAVE_EVP_AES_CTR # define HAVE_AEAD @@ -303,6 +313,7 @@ # ifdef RSA_PKCS1_PSS_PADDING # define HAVE_RSA_PKCS1_PSS_PADDING # endif +# define HAS_PKCS5_PBKDF2_HMAC #endif #if OPENSSL_VERSION_NUMBER >= PACKED_OPENSSL_VERSION(0,9,8,'h') \ @@ -428,11 +439,17 @@ do { \ #endif -#ifdef FIPS_SUPPORT -/* In FIPS mode non-FIPS algorithms are disabled and return badarg. */ -#define CHECK_NO_FIPS_MODE() { if (FIPS_mode()) return atom_notsup; } +/* This is not the final FIPS adaptation for 3.0, just making it compilable */ +#if OPENSSL_VERSION_NUMBER >= PACKED_OPENSSL_VERSION_PLAIN(3,0,0) +# undef FIPS_SUPPORT +#endif + +#if defined(FIPS_SUPPORT) +# define FIPS_MODE() (FIPS_mode() ? 1 : 0) #else -#define CHECK_NO_FIPS_MODE() +# define FIPS_MODE() 0 #endif + + #endif /* E_OPENSSL_CONFIG_H__ */ diff --git a/lib/crypto/c_src/openssl_version.h b/lib/crypto/c_src/openssl_version.h new file mode 100644 index 0000000000..5ed36ef9d6 --- /dev/null +++ b/lib/crypto/c_src/openssl_version.h @@ -0,0 +1,52 @@ +/* + * %CopyrightBegin% + * + * Copyright Ericsson AB 2010-2020. All Rights Reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + * + * %CopyrightEnd% + */ + +#ifndef E_OPENSSL_VERSION_H__ +#define E_OPENSSL_VERSION_H__ 1 + +#include <openssl/opensslv.h> + +#ifdef LIBRESSL_VERSION_NUMBER +# define HAS_LIBRESSL +#endif + +/* Helper macros to construct a OPENSSL_VERSION_NUMBER. + * See openssl/opensslv.h + */ + +#if !defined(HAS_LIBRESSL) && \ + defined(OPENSSL_VERSION_MAJOR) && \ + (OPENSSL_VERSION_MAJOR >= 3) + +# define PACKED_OPENSSL_VERSION(MAJ, MIN, PATCH, VOID) \ + (((((MAJ << 8) | MIN) << 16 ) | PATCH) << 4) +#else +/* Pre 3.0.0 */ +# define PACKED_OPENSSL_VERSION(MAJ, MIN, FIX, P) \ + ((((((((MAJ << 8) | MIN) << 8 ) | FIX) << 8) | (P-'a'+1)) << 4) | 0xf) + +/* End Pre 3.0.0 */ +#endif + +#define PACKED_OPENSSL_VERSION_PLAIN(MAJ, MIN, FIX) \ + PACKED_OPENSSL_VERSION(MAJ,MIN,FIX,('a'-1)) + + +#endif /* E_OPENSSL_VERSION_H__ */ diff --git a/lib/crypto/c_src/pbkdf2_hmac.c b/lib/crypto/c_src/pbkdf2_hmac.c new file mode 100644 index 0000000000..7eec603d9c --- /dev/null +++ b/lib/crypto/c_src/pbkdf2_hmac.c @@ -0,0 +1,75 @@ +/* + * %CopyrightBegin% + * + * Copyright Ericsson AB 2010-2021. All Rights Reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + * + * %CopyrightEnd% + */ + +#include "common.h" +#include "pbkdf2_hmac.h" +#include "digest.h" + +ERL_NIF_TERM pbkdf2_hmac_nif(ErlNifEnv* env, int argc, + const ERL_NIF_TERM argv[]) +{ +#ifdef HAS_PKCS5_PBKDF2_HMAC + ErlNifBinary pass, salt, out; + ErlNifUInt64 iter, keylen; + struct digest_type_t* digp = NULL; + + ASSERT(argc == 5); + + if ((digp = get_digest_type(argv[0])) == NULL) + goto bad_arg; + if (digp->md.p == NULL) + goto bad_arg; + if ((digp->flags & PBKDF2_ELIGIBLE_DIGEST) == 0) { + goto bad_arg; + } + + if (!enif_inspect_binary(env, argv[1], &pass)) + goto bad_arg; + if (!enif_inspect_binary(env, argv[2], &salt)) + goto bad_arg; + if (!enif_get_uint64(env, argv[3], &iter)) + goto bad_arg; + if (!enif_get_uint64(env, argv[4], &keylen)) + goto bad_arg; + + if (iter < 1) + goto bad_arg; + if (keylen < 1) + goto bad_arg; + + if (!enif_alloc_binary(keylen, &out)) + goto err; + + if (!PKCS5_PBKDF2_HMAC((const char *)pass.data, pass.size, + salt.data, salt.size, iter, + digp->md.p, + keylen, out.data)) { + enif_release_binary(&out); + goto err; + } + + return enif_make_binary(env, &out); + bad_arg: + err: + return enif_make_badarg(env); +#else + return EXCP_NOTSUP(env, "Unsupported CRYPTO_PKCS5_PBKDF2_HMAC"); +#endif +} diff --git a/lib/crypto/c_src/pbkdf2_hmac.h b/lib/crypto/c_src/pbkdf2_hmac.h new file mode 100644 index 0000000000..90be790861 --- /dev/null +++ b/lib/crypto/c_src/pbkdf2_hmac.h @@ -0,0 +1,29 @@ +/* + * %CopyrightBegin% + * + * Copyright Ericsson AB 2010-2021. All Rights Reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + * + * %CopyrightEnd% + */ + +#ifndef E_PBKDF2_HMAC_H__ +#define E_PBKDF2_HMAC_H__ 1 + +#include "common.h" + +ERL_NIF_TERM pbkdf2_hmac_nif(ErlNifEnv* env, int argc, + const ERL_NIF_TERM argv[]); + +#endif /* E_PBKDF2_HMAC_H__ */ diff --git a/lib/crypto/c_src/pkey.c b/lib/crypto/c_src/pkey.c index 2125aed537..2c2fbc9a15 100644 --- a/lib/crypto/c_src/pkey.c +++ b/lib/crypto/c_src/pkey.c @@ -73,7 +73,7 @@ static int get_pkey_digest_type(ErlNifEnv *env, ERL_NIF_TERM algorithm, ERL_NIF_ return PKEY_OK; if (algorithm == atom_eddsa) { #ifdef HAVE_EDDSA - if (!FIPS_mode()) return PKEY_OK; + if (!FIPS_MODE()) return PKEY_OK; #else return PKEY_NOTSUP; #endif @@ -282,10 +282,10 @@ static int get_pkey_private_key(ErlNifEnv *env, ERL_NIF_TERM algorithm, ERL_NIF_ #else return PKEY_BADARG; #endif + } else if (algorithm == atom_rsa) { if ((rsa = RSA_new()) == NULL) goto err; - if (!get_rsa_private_key(env, key, rsa)) goto err; if ((result = EVP_PKEY_new()) == NULL) @@ -299,7 +299,6 @@ static int get_pkey_private_key(ErlNifEnv *env, ERL_NIF_TERM algorithm, ERL_NIF_ #if defined(HAVE_EC) const ERL_NIF_TERM *tpl_terms; int tpl_arity; - if (!enif_get_tuple(env, key, &tpl_arity, &tpl_terms)) goto err; if (tpl_arity != 2) @@ -317,48 +316,41 @@ static int get_pkey_private_key(ErlNifEnv *env, ERL_NIF_TERM algorithm, ERL_NIF_ goto err; /* On success, result owns ec */ ec = NULL; - #else return PKEY_NOTSUP; #endif + } else if (algorithm == atom_eddsa) { #ifdef HAVE_EDDSA - if (!FIPS_mode()) - { - if (!get_eddsa_key(env, 0, key, &result)) - goto err; - else - goto done; // Not nice.... - } + if (FIPS_MODE()) + return PKEY_NOTSUP; + if (!get_eddsa_key(env, 0, key, &result)) + goto err; #else return PKEY_NOTSUP; #endif + } else if (algorithm == atom_dss) { #ifdef HAVE_DSA if ((dsa = DSA_new()) == NULL) goto err; if (!get_dss_private_key(env, key, dsa)) goto err; - if ((result = EVP_PKEY_new()) == NULL) goto err; if (EVP_PKEY_assign_DSA(result, dsa) != 1) goto err; /* On success, result owns dsa */ dsa = NULL; - - } else { +#else + return PKEY_NOTSUP; #endif + + } else return PKEY_BADARG; - } - goto done; - err: - if (result) - EVP_PKEY_free(result); - result = NULL; - done: + free_and_return: if (password) enif_free(password); if (id) @@ -380,8 +372,15 @@ static int get_pkey_private_key(ErlNifEnv *env, ERL_NIF_TERM algorithm, ERL_NIF_ *pkey = result; return PKEY_OK; } + + err: + if (result) + EVP_PKEY_free(result); + result = NULL; + goto free_and_return; } + static int get_pkey_public_key(ErlNifEnv *env, ERL_NIF_TERM algorithm, ERL_NIF_TERM key, EVP_PKEY **pkey) { @@ -453,7 +452,7 @@ static int get_pkey_public_key(ErlNifEnv *env, ERL_NIF_TERM algorithm, ERL_NIF_T #endif } else if (algorithm == atom_eddsa) { #ifdef HAVE_EDDSA - if (!FIPS_mode()) { + if (!FIPS_MODE()) { if (!get_eddsa_key(env, 1, key, &result)) goto err; } @@ -610,7 +609,7 @@ enif_get_atom(env,argv[1],buf,1024,ERL_NIF_LATIN1); printf("hash=%s ",buf); if (argv[0] == atom_eddsa) { #ifdef HAVE_EDDSA - if (!FIPS_mode()) { + if (!FIPS_MODE()) { if ((mdctx = EVP_MD_CTX_new()) == NULL) goto err; @@ -843,7 +842,7 @@ ERL_NIF_TERM pkey_verify_nif(ErlNifEnv *env, int argc, const ERL_NIF_TERM argv[] if (argv[0] == atom_eddsa) { #ifdef HAVE_EDDSA - if (!FIPS_mode()) { + if (!FIPS_MODE()) { if ((mdctx = EVP_MD_CTX_new()) == NULL) goto err; diff --git a/lib/crypto/c_src/srp.c b/lib/crypto/c_src/srp.c index 22fbedb4eb..330c579d47 100644 --- a/lib/crypto/c_src/srp.c +++ b/lib/crypto/c_src/srp.c @@ -30,7 +30,7 @@ ERL_NIF_TERM srp_value_B_nif(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[] int dlen; ERL_NIF_TERM ret; - CHECK_NO_FIPS_MODE(); + if (FIPS_MODE()) return atom_notsup; ASSERT(argc == 5); @@ -122,7 +122,7 @@ ERL_NIF_TERM srp_user_secret_nif(ErlNifEnv* env, int argc, const ERL_NIF_TERM ar int dlen; ERL_NIF_TERM ret; - CHECK_NO_FIPS_MODE(); + if (FIPS_MODE()) return atom_notsup; ASSERT(argc == 7); @@ -232,7 +232,7 @@ ERL_NIF_TERM srp_host_secret_nif(ErlNifEnv* env, int argc, const ERL_NIF_TERM ar int dlen; ERL_NIF_TERM ret; - CHECK_NO_FIPS_MODE(); + if (FIPS_MODE()) return atom_notsup; ASSERT(argc == 5); diff --git a/lib/crypto/configure b/lib/crypto/configure index 82e882c337..fdeba8fc08 100755 --- a/lib/crypto/configure +++ b/lib/crypto/configure @@ -747,6 +747,7 @@ with_ssl_rpath enable_dynamic_ssl_lib enable_evp_dh enable_evp_hmac +enable_deprecated_warnings enable_fips ' ac_precious_vars='build_alias @@ -1419,6 +1420,10 @@ Optional Features: the crypto NIF --disable-evp-dh intentionally undocumented workaround --disable-evp-hmac intentionally undocumented workaround + --disable-deprecated-warnings + disable warnings for deprecated functions in + cryptolib (default is to warn, except for OpenSSL + 3.x where the default is not to warn) --enable-fips enable OpenSSL FIPS mode support --disable-fips disable OpenSSL FIPS mode support (default) @@ -5091,6 +5096,17 @@ else fi +# Check whether --enable-deprecated_warnings was given. +if test "${enable_deprecated_warnings+set}" = set; then : + enableval=$enable_deprecated_warnings; case "$enableval" in + no) deprecated_warnings=no;; + *) deprecated_warnings=yes;; + esac +else + deprecated_warnings=default_yes +fi + + @@ -5251,6 +5267,23 @@ else fi rm -f conftest* + cat confdefs.h - <<_ACEOF >conftest.$ac_ext +/* end confdefs.h. */ + +#include <openssl/opensslv.h> +#if OPENSSL_VERSION_MAJOR == 3 +yes +#endif + +_ACEOF +if (eval "$ac_cpp conftest.$ac_ext") 2>&5 | + $EGREP "^yes.?$" >/dev/null 2>&1; then : + v3_include=yes +else + v3_include=no +fi +rm -f conftest* + CPPFLAGS="$save_CPPFLAGS" if test $valid_include != yes; then @@ -5639,6 +5672,23 @@ else fi rm -f conftest* + cat confdefs.h - <<_ACEOF >conftest.$ac_ext +/* end confdefs.h. */ + +#include <openssl/opensslv.h> +#if OPENSSL_VERSION_MAJOR == 3 +yes +#endif + +_ACEOF +if (eval "$ac_cpp conftest.$ac_ext") 2>&5 | + $EGREP "^yes.?$" >/dev/null 2>&1; then : + v3_include=yes +else + v3_include=no +fi +rm -f conftest* + CPPFLAGS="$save_CPPFLAGS" if test $valid_include != yes; then @@ -6259,6 +6309,14 @@ else SSL_FLAGS= fi +if test "x$v3_include" = "xyes" && test "x$deprecated_warnings" = "xdefault_yes" && test "$CRYPTO_APP" != ""; then + deprecated_warnings=no +fi + +if test "x$deprecated_warnings" = "xno" && test "$CRYPTO_APP" != "" ; then + SSL_FLAGS="$SSL_FLAGS -Wno-deprecated-declarations" +fi + diff --git a/lib/crypto/configure.in b/lib/crypto/configure.in index e71b2482c6..fd337b3d34 100644 --- a/lib/crypto/configure.in +++ b/lib/crypto/configure.in @@ -156,6 +156,14 @@ AS_HELP_STRING([--disable-evp-hmac], *) DISABLE_EVP_HMAC=0;; esac ], DISABLE_EVP_HMAC=0) +AC_ARG_ENABLE(deprecated_warnings, +AS_HELP_STRING([--disable-deprecated-warnings], + [disable warnings for deprecated functions in cryptolib (default is to warn, except for OpenSSL 3.x where the default is not to warn)]), +[ case "$enableval" in + no) deprecated_warnings=no;; + *) deprecated_warnings=yes;; + esac ], deprecated_warnings=default_yes) + AC_DEFUN([ERL_LINK_CRYPTO_IFELSE], [ test_cflags=$1 test_ldflags=$2 @@ -220,6 +228,14 @@ yes ], [valid_include=yes], [valid_include=no]) + AC_EGREP_CPP(^yes.?$,[ +#include <openssl/opensslv.h> +#if OPENSSL_VERSION_MAJOR == 3 +yes +#endif + ], + [v3_include=yes], + [v3_include=no]) CPPFLAGS="$save_CPPFLAGS" if test $valid_include != yes; then @@ -791,6 +807,14 @@ else SSL_FLAGS= fi +if test "x$v3_include" = "xyes" && test "x$deprecated_warnings" = "xdefault_yes" && test "$CRYPTO_APP" != ""; then + deprecated_warnings=no +fi + +if test "x$deprecated_warnings" = "xno" && test "$CRYPTO_APP" != "" ; then + SSL_FLAGS="$SSL_FLAGS -Wno-deprecated-declarations" +fi + AC_SUBST(SSL_INCLUDE) AC_SUBST(SSL_INCDIR) AC_SUBST(SSL_LIBDIR) diff --git a/lib/crypto/doc/src/crypto.xml b/lib/crypto/doc/src/crypto.xml index 9a38ec53f4..92c9cb43d3 100644 --- a/lib/crypto/doc/src/crypto.xml +++ b/lib/crypto/doc/src/crypto.xml @@ -1153,6 +1153,27 @@ </func> <func> + <name name="info" arity="0" since="OTP @OTP-17603@"/> + <fsummary>Provides information about crypto and the library used by crypto.</fsummary> + <desc> + <p>Provides a map with information about the compilation and linking of crypto. + </p> + <p>Example:</p> + <code> +1> crypto:info(). +#{compile_type => normal, + cryptolib_version_compiled => "OpenSSL 3.0.0 7 sep 2021", + cryptolib_version_linked => "OpenSSL 3.0.0 7 sep 2021", + link_type => dynamic, + otp_crypto_version => "5.0.2"} +2> + </code> + <p>More association types than documented may be present in the map. + </p> + </desc> + </func> + + <func> <name name="info_lib" arity="0" since=""/> <fsummary>Provides information about the libraries used by crypto.</fsummary> <desc> @@ -1938,6 +1959,19 @@ FloatValue = rand:uniform(). % again </desc> </func> + <func> + <name name="pbkdf2_hmac" arity="5" since="OTP @OTP-17808@"/> + <fsummary>PBKDF2 in combination with HMAC</fsummary> + <desc> + <p> + PKCS #5 PBKDF2 (Password-Based Key Derivation Function 2) in combination with HMAC. + </p> + <p> + The function raises a <c>error:badarg</c> if the parameters are in wrong format. + </p> + </desc> + </func> + </funcs> </erlref> diff --git a/lib/crypto/doc/src/crypto_app.xml b/lib/crypto/doc/src/crypto_app.xml index 3b9d9f1a8e..de361f2fd2 100644 --- a/lib/crypto/doc/src/crypto_app.xml +++ b/lib/crypto/doc/src/crypto_app.xml @@ -49,6 +49,13 @@ version that is officially supported by the OpenSSL project. API compatible backends like LibreSSL should also work.</p> + <p>The crypto app is tested daily with at least one version of each of the + OpenSSL 0.9.8, 1.0.0, 1.0.1, 1.0.2, 1.1.0 and 1.1.1. FIPS mode is also tested.</p> + + <note><p>Compiling, linking and running with OpenSSL 3.0.0 works although the + crypto app calls deprecated functions. We do not recommend it for other than + experimental purposes or alpha testing, since it is not extensively tested yet.</p></note> + <p>Source releases of OpenSSL can be downloaded from the <url href="http://www.openssl.org">OpenSSL</url> project home page, or mirror sites listed there. </p> diff --git a/lib/crypto/src/crypto.erl b/lib/crypto/src/crypto.erl index 473be885c9..1d1ad71639 100644 --- a/lib/crypto/src/crypto.erl +++ b/lib/crypto/src/crypto.erl @@ -43,6 +43,8 @@ -export([ec_curve/1, ec_curves/0]). -export([rand_seed/1]). -export([format_error/2]). +-export([pbkdf2_hmac/5]). + %%%---------------------------------------------------------------- %% Removed functions. %% @@ -526,7 +528,7 @@ supports() -> supports(hashs) -> hash_algorithms(); supports(public_keys) -> pubkey_algorithms(); -supports(ciphers) -> cipher_algorithms(); +supports(ciphers) -> add_cipher_aliases(cipher_algorithms()); supports(macs) -> mac_algorithms(); supports(curves) -> curve_algorithms(); supports(rsa_opts) -> rsa_opts_algorithms(). @@ -537,8 +539,15 @@ supports(rsa_opts) -> rsa_opts_algorithms(). VerStr :: binary() . info_lib() -> ?nif_stub. -%% info/0 and info_nif/0 are experimental. May disapear or change without warning. -info() -> info_nif(). +-spec info() -> #{compile_type := normal | debug | valgrind | asan, + cryptolib_version_compiled => string() | undefined, + cryptolib_version_linked := string(), + link_type := dynamic | static, + otp_crypto_version := string() + }. +info() -> + (info_nif())#{otp_crypto_version => crypto:version()}. + info_nif() -> ?nif_stub. @@ -553,6 +562,18 @@ enable_fips_mode(Enable) -> enable_fips_mode_nif(_) -> ?nif_stub. +-spec pbkdf2_hmac(Digest, Pass, Salt, Iter, KeyLen) -> Result + when Digest :: sha | sha224 | sha256 | sha384 | sha512, + Pass :: binary(), + Salt :: binary(), + Iter :: pos_integer(), + KeyLen :: pos_integer(), + Result :: binary(). +pbkdf2_hmac(Digest, Pass, Salt, Iter, KeyLen) -> + pbkdf2_hmac_nif(Digest, Pass, Salt, Iter, KeyLen). + +pbkdf2_hmac_nif(_, _, _, _, _) -> ?nif_stub. + %%%================================================================ %%% %%% Hashing @@ -985,6 +1006,16 @@ cipher_info_nif(_Type) -> ?nif_stub. %%% Cipher aliases %%% +add_cipher_aliases(Ciphers) -> + Ciphers ++ + lists:usort( + lists:foldl(fun(C, Acc) -> + case alias1_rev(C) of + C -> Acc; + A -> [A|Acc] + end + end, [], Ciphers)). + alias(aes_cbc, Key) -> alias1(aes_cbc, iolist_size(Key)); alias(aes_cfb8, Key) -> alias1(aes_cfb8, iolist_size(Key)); alias(aes_cfb128, Key) -> alias1(aes_cfb128, iolist_size(Key)); @@ -1025,6 +1056,37 @@ alias1(aes_ccm, 32) -> aes_256_ccm; alias1(Alg, _) -> Alg. + +alias1_rev(aes_128_cbc) -> aes_cbc; +alias1_rev(aes_192_cbc) -> aes_cbc; +alias1_rev(aes_256_cbc) -> aes_cbc; + +alias1_rev(aes_128_cfb8) -> aes_cfb8; +alias1_rev(aes_192_cfb8) -> aes_cfb8; +alias1_rev(aes_256_cfb8) -> aes_cfb8; + +alias1_rev(aes_128_cfb128) -> aes_cfb128; +alias1_rev(aes_192_cfb128) -> aes_cfb128; +alias1_rev(aes_256_cfb128) -> aes_cfb128; + +alias1_rev(aes_128_ctr) -> aes_ctr; +alias1_rev(aes_192_ctr) -> aes_ctr; +alias1_rev(aes_256_ctr) -> aes_ctr; + +alias1_rev(aes_128_ecb) -> aes_ecb; +alias1_rev(aes_192_ecb) -> aes_ecb; +alias1_rev(aes_256_ecb) -> aes_ecb; + +alias1_rev(aes_128_gcm) -> aes_gcm; +alias1_rev(aes_192_gcm) -> aes_gcm; +alias1_rev(aes_256_gcm) -> aes_gcm; + +alias1_rev(aes_128_ccm) -> aes_ccm; +alias1_rev(aes_192_ccm) -> aes_ccm; +alias1_rev(aes_256_ccm) -> aes_ccm; + +alias1_rev(C) -> C. + %%%================================================================ %%% %%% RAND - pseudo random numbers using RN_ and BN_ functions in crypto lib @@ -1560,13 +1622,14 @@ generate_key(rsa, {ModulusSize, PublicExponent}, undefined) -> {lists:sublist(Private, 2), Private} end; -generate_key(eddh, Curve, PrivKey) when Curve == x448 ; - Curve == x25519 -> - evp_generate_key_nif(Curve, ensure_int_as_bin(PrivKey)); generate_key(ecdh, Curve, PrivKey) when Curve == x448 ; Curve == x25519 -> - %% This was here before the eddh was added as an own Type + %% Legacy: This clause was here before the eddh was added as an own Type + generate_key(eddh, Curve, PrivKey); +generate_key(eddh, Curve, PrivKey) when Curve == x448 ; + Curve == x25519 -> evp_generate_key_nif(Curve, ensure_int_as_bin(PrivKey)); + generate_key(ecdh, Curve, PrivKey) -> ec_key_generate(nif_curve_params(Curve), ensure_int_as_bin(PrivKey)); @@ -1627,7 +1690,8 @@ compute_key(srp, UserPublic, {HostPublic, HostPrivate}, compute_key(ecdh, Others, My, Curve) when Curve == x448 ; Curve == x25519 -> - evp_compute_key_nif(Curve, ensure_int_as_bin(Others), ensure_int_as_bin(My)); + %% Legacy: This clause was here before the eddh was added as an own Type + compute_key(eddh, Others, My, Curve); compute_key(eddh, Others, My, Curve) when Curve == x448 ; Curve == x25519 -> diff --git a/lib/crypto/test/crypto_SUITE.erl b/lib/crypto/test/crypto_SUITE.erl index 2f91f7037a..44c74a70a7 100644 --- a/lib/crypto/test/crypto_SUITE.erl +++ b/lib/crypto/test/crypto_SUITE.erl @@ -110,6 +110,8 @@ use_all_ec_sign_verify/1, use_all_ecdh_generate_compute/1, use_all_eddh_generate_compute/1, + pbkdf2_hmac/0, + pbkdf2_hmac/1, %% Others: aes_128_cbc/1, @@ -191,7 +193,8 @@ all() -> rand_plugin_s, info, cipher_info, - hash_info + hash_info, + pbkdf2_hmac ]. -define(NEW_CIPHER_TYPE_SCHEMA, @@ -448,10 +451,11 @@ init_per_suite(Config) -> try is_ok(crypto:start()) of ok -> catch ct:comment("~s",[element(3,hd(crypto:info_lib()))]), - catch ct:log("crypto:info_lib() -> ~p~n" - "crypto:supports() -> ~p~n" + catch ct:log("crypto:info() -> ~p~n" + "crypto:info_lib() -> ~p~n" "crypto:version() -> ~p~n" - ,[crypto:info_lib(), crypto:supports(), crypto:version()]), + "crypto:supports() -> ~p~n" + ,[crypto:info(), crypto:info_lib(), crypto:version(), crypto:supports()]), try crypto:strong_rand_bytes(1) of _ -> @@ -4346,3 +4350,62 @@ try_enable_fips_mode(Config) -> not_supported -> {skip, "FIPS mode not supported"} end. + +pbkdf2_hmac() -> + [{doc, "Test the pbkdf2_hmac function"}]. +pbkdf2_hmac(Config) when is_list(Config) -> + try + F = fun (A, B, C, D) -> + binary:encode_hex(crypto:pbkdf2_hmac(sha, A, B, C, D)) + end, + %% RFC 6070 + <<"0C60C80F961F0E71F3A9B524AF6012062FE037A6">> = + F(<<"password">>, <<"salt">>, 1, 20), + <<"EA6C014DC72D6F8CCD1ED92ACE1D41F0D8DE8957">> = + F(<<"password">>, <<"salt">>, 2, 20), + <<"4B007901B765489ABEAD49D926F721D065A429C1">> = + F(<<"password">>, <<"salt">>, 4096, 20), + <<"EEFE3D61CD4DA4E4E9945B3D6BA2158C2634E984">> = + F(<<"password">>, <<"salt">>, 16777216, 20), + <<"3D2EEC4FE41C849B80C8D83662C0E44A8B291A964CF2F07038">> = + F(<<"passwordPASSWORDpassword">>, <<"saltSALTsaltSALTsaltSALTsaltSALTsalt">>, 4096, 25), + <<"56FA6AA75548099DCC37D7F03425E0C3">> = + F(<<"pass\0word">>, <<"sa\0lt">>, 4096, 16), + + %% RFC 3962 + <<"CDEDB5281BB2F801565A1122B2563515">> = + F(<<"password">>, <<"ATHENA.MIT.EDUraeburn">>, 1, 16), + <<"CDEDB5281BB2F801565A1122B25635150AD1F7A04BB9F3A333ECC0E2E1F70837">> = + F(<<"password">>, <<"ATHENA.MIT.EDUraeburn">>, 1, 32), + <<"01DBEE7F4A9E243E988B62C73CDA935D">> = + F(<<"password">>, <<"ATHENA.MIT.EDUraeburn">>, 2, 16), + <<"01DBEE7F4A9E243E988B62C73CDA935DA05378B93244EC8F48A99E61AD799D86">> = + F(<<"password">>, <<"ATHENA.MIT.EDUraeburn">>, 2, 32), + <<"5C08EB61FDF71E4E4EC3CF6BA1F5512B">> = + F(<<"password">>, <<"ATHENA.MIT.EDUraeburn">>, 1200, 16), + <<"5C08EB61FDF71E4E4EC3CF6BA1F5512BA7E52DDBC5E5142F708A31E2E62B1E13">> = + F(<<"password">>, <<"ATHENA.MIT.EDUraeburn">>, 1200, 32), + <<"D1DAA78615F287E6A1C8B120D7062A49">> = + F(<<"password">>, binary:encode_unsigned(16#1234567878563412), 5, 16), + <<"D1DAA78615F287E6A1C8B120D7062A493F98D203E6BE49A6ADF4FA574B6E64EE">> = + F(<<"password">>, binary:encode_unsigned(16#1234567878563412), 5, 32), + <<"139C30C0966BC32BA55FDBF212530AC9">> = + F(<<"XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX">>, + <<"pass phrase equals block size">>, 1200, 16), + <<"139C30C0966BC32BA55FDBF212530AC9C5EC59F1A452F5CC9AD940FEA0598ED1">> = + F(<<"XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX">>, + <<"pass phrase equals block size">>, 1200, 32), + <<"9CCAD6D468770CD51B10E6A68721BE61">> = + F(<<"XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX">>, + <<"pass phrase exceeds block size">>, 1200, 16), + <<"9CCAD6D468770CD51B10E6A68721BE611A8B4D282601DB3B36BE9246915EC82A">> = + F(<<"XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX">>, + <<"pass phrase exceeds block size">>, 1200, 32), + <<"6B9CF26D45455A43A5B8BB276A403B39">> = + F(binary:encode_unsigned(16#f09d849e), <<"EXAMPLE.COMpianist">>, 50, 16), + <<"6B9CF26D45455A43A5B8BB276A403B39E7FE37A0C41E02C281FF3069E1E94F52">> = + F(binary:encode_unsigned(16#f09d849e), <<"EXAMPLE.COMpianist">>, 50, 32) + catch + error:{notsup, _, "Unsupported CRYPTO_PKCS5_PBKDF2_HMAC"} -> + {skip, "No CRYPTO_PKCS5_PBKDF2_HMAC"} + end. diff --git a/lib/crypto/test/engine_SUITE.erl b/lib/crypto/test/engine_SUITE.erl index e75485f2bf..677b0c35f8 100644 --- a/lib/crypto/test/engine_SUITE.erl +++ b/lib/crypto/test/engine_SUITE.erl @@ -139,6 +139,9 @@ init_per_suite(Config) -> {_, [{_,_, <<"OpenSSL 1.0.1s-freebsd 1 Mar 2016">>}]} -> {skip, "Problem with engine on OpenSSL 1.0.1s-freebsd"}; + {_, [{_,_,<<"LibreSSL 2.1.",_/binary>>}]} -> + {skip, "Problem with engine on older LibreSSL 2.1.*"}; + {{unix,darwin}, _} -> {skip, "Engine unsupported on Darwin"}; diff --git a/lib/edoc/src/edoc.erl b/lib/edoc/src/edoc.erl index 6ec5565fb0..b1cf537f07 100644 --- a/lib/edoc/src/edoc.erl +++ b/lib/edoc/src/edoc.erl @@ -286,7 +286,8 @@ opt_defaults() -> opt_negations() -> [{no_preprocess, preprocess}, {no_subpackages, subpackages}, - {no_report_missing_types, report_missing_types}]. + {no_report_missing_types, report_missing_types}, + {no_link_predefined_types, link_predefined_types}]. %% @doc Runs EDoc on a given set of source files. Note %% that the doclet plugin module has its own particular options; see the @@ -613,6 +614,15 @@ read_source(Name) -> %% `no_report_missing_types' is an alias for %% `{report_missing_types, false}'. %% </dd> +%% <dt>{@type {link_predefined_types, boolean()@}} +%% </dt> +%% <dd>If the value is `true', all predefined data types will have a link +%% to the erlang module. This option is to be used when generating +%% documentation for the Erlang/OTP docs. +%% The default value is `false'. +%% `no_link_predefined_types' is an alias for +%% `{link_predefined_types, false}'. +%% </dd> %% </dl> %% %% @see get_doc/2 diff --git a/lib/edoc/src/edoc_data.erl b/lib/edoc/src/edoc_data.erl index ae77ed0bb3..30b6e3e0c9 100644 --- a/lib/edoc/src/edoc_data.erl +++ b/lib/edoc/src/edoc_data.erl @@ -31,7 +31,7 @@ -module(edoc_data). --export([module/4, overview/4, type/2]). +-export([module/4, overview/4, type/3]). -export([hidden_filter/2, get_all_tags/1, get_entry/2, get_tags/2]). @@ -117,7 +117,7 @@ module(Module, Entries, Env, Opts) -> ++ sees(HeaderTags, Env) ++ references(HeaderTags) ++ todos(HeaderTags, Opts) - ++ [{typedecls, types(AllTags, Env)}, + ++ [{typedecls, types(AllTags, Env, Opts)}, {functions, functions(Functions, Env, Opts)} | callbacks(Functions, Module, Env, Opts)]) }, @@ -142,9 +142,9 @@ module_args(none) -> module_args(Vs) -> [{args, [{arg, [{argName, [atom_to_list(V)]}]} || V <- Vs]}]. -types(Tags, Env) -> +types(Tags, Env, Opts) -> [{typedecl, [{label, edoc_types:to_label(Def)}, {line, Line}], - [edoc_types:to_xml(Def, Env)] ++ description(Doc)} + [edoc_types:to_xml(Def, Env, Opts)] ++ description(Doc)} || #tag{name = type, line = Line, data = {Def, Doc}} <- Tags]. functions(Es, Env, Opts) -> @@ -250,9 +250,9 @@ function({N, A}, As0, Export, Ts, Env, Opts) -> {private, yes_or_no(is_private(Ts))}, {hidden, yes_or_no(is_hidden(Ts))}, {label, edoc_refs:to_label(edoc_refs:function(N, A))}], - lists:append([get_args(lists:nth(Clause, As0), Ts, Clause, Env) + lists:append([get_args(lists:nth(Clause, As0), Ts, Clause, Env, Opts) || Clause <- lists:seq(1, length(As0))]) - ++ get_throws(Ts, Env) + ++ get_throws(Ts, Env, Opts) ++ get_equiv(Ts, Env) ++ get_doc(Ts) ++ get_since(Ts) @@ -261,8 +261,8 @@ function({N, A}, As0, Export, Ts, Env, Opts) -> ++ todos(Ts, Opts) }. -get_args(As, Ts, Clause, Env) -> - {Args, Ret, Spec} = signature(Ts, As, Clause, Env), +get_args(As, Ts, Clause, Env, Opts) -> + {Args, Ret, Spec} = signature(Ts, As, Clause, Env, Opts), [{args, [{arg, [{argName, [atom_to_list(A)]}] ++ description(D)} || {A, D} <- Args]}] ++ Spec @@ -274,11 +274,11 @@ get_args(As, Ts, Clause, Env) -> yes_or_no(true) -> "yes"; yes_or_no(false) -> "no". -get_throws(Ts, Env) -> +get_throws(Ts, Env, Opts) -> case get_tags(throws, Ts) of [Throws] -> Type = Throws#tag.data, - [edoc_types:to_xml(Type, Env)]; + [edoc_types:to_xml(Type, Env, Opts)]; [] -> [] end. @@ -427,7 +427,7 @@ todos(Tags, Opts) -> [] end. -signature(Ts, As, Clause, Env) -> +signature(Ts, As, Clause, Env, Opts) -> case get_tags(spec, Ts) of [T] -> Spec = maybe_nth(Clause, T#tag.data), @@ -439,7 +439,7 @@ signature(Ts, As, Clause, Env) -> As1 = merge_args(As0, As, Ds0, P), %% check_params(As1, P), Spec1 = edoc_types:set_arg_names(Spec, [A || {A,_} <- As1]), - {As1, R, [edoc_types:to_xml(Spec1, Env)]}; + {As1, R, [edoc_types:to_xml(Spec1, Env, Opts)]}; [] -> S = sets:new(), {[{A, ""} || A <- fix_argnames(As, S, 1)], [], []} @@ -523,8 +523,8 @@ get_tags(_, []) -> []. %% --------------------------------------------------------------------- -type(T, Env) -> - xmerl_lib:expand_element({type, [edoc_types:to_xml(T, Env)]}). +type(T, Env, Opts) -> + xmerl_lib:expand_element({type, [edoc_types:to_xml(T, Env, Opts)]}). %% <!ELEMENT overview (title, description?, author*, copyright?, version?, %% since?, see*, reference*, todo?, modules)> diff --git a/lib/edoc/src/edoc_macros.erl b/lib/edoc/src/edoc_macros.erl index 9fed6797e9..899a59c0d6 100644 --- a/lib/edoc/src/edoc_macros.erl +++ b/lib/edoc/src/edoc_macros.erl @@ -103,7 +103,7 @@ type_macro(S, Line, Env) -> S1 = "t()=" ++ S, Def = edoc_parser:parse_typedef(S1, Line), {#t_typedef{type = T}, _} = Def, - Txt = edoc_layout:type(edoc_data:type(T, Env)), + Txt = edoc_layout:type(edoc_data:type(T, Env, [])), lists:flatten(io_lib:fwrite("<code>~ts</code>", [Txt])). diff --git a/lib/edoc/src/edoc_types.erl b/lib/edoc/src/edoc_types.erl index 9da447bec5..fd427b2c1a 100644 --- a/lib/edoc/src/edoc_types.erl +++ b/lib/edoc/src/edoc_types.erl @@ -31,7 +31,7 @@ -module(edoc_types). -export([is_predefined/2, is_new_predefined/2, - to_ref/1, to_xml/2, to_label/1, arg_names/1, set_arg_names/2, + to_ref/1, to_xml/3, to_label/1, arg_names/1, set_arg_names/2, arg_descs/1, range_desc/1]). %% @headerfile "edoc_types.hrl" @@ -172,18 +172,18 @@ infer_module_app(#t_name{app = [], module = M} = TName) when is_atom(M) -> infer_module_app(Other) -> Other. -to_xml(#t_var{name = N}, _Env) -> +to_xml(#t_var{name = N}, _Env, _Opts) -> {typevar, [{name, atom_to_list(N)}], []}; -to_xml(#t_name{module = [], name = N}, _Env) -> +to_xml(#t_name{module = [], name = N}, _Env, _Opts) -> {erlangName, [{name, atom_to_list(N)}], []}; -to_xml(#t_name{app = [], module = M, name = N}, _Env) -> +to_xml(#t_name{app = [], module = M, name = N}, _Env, _Opts) -> {erlangName, [{module, atom_to_list(M)}, {name, atom_to_list(N)}], []}; -to_xml(#t_name{app = A, module = M, name = N}, _Env) -> +to_xml(#t_name{app = A, module = M, name = N}, _Env, _Opts) -> {erlangName, [{app, atom_to_list(A)}, {module, atom_to_list(M)}, {name, atom_to_list(N)}], []}; -to_xml(#t_type{name = N, args = As}, Env) -> +to_xml(#t_type{name = N, args = As}, Env, Opts) -> Predef = case N of #t_name{module = [], name = T} -> NArgs = length(As), @@ -191,86 +191,89 @@ to_xml(#t_type{name = N, args = As}, Env) -> _ -> false end, - HRef = case Predef of - true -> []; - false -> [{href, get_uri(N, Env)}] + HRef = case {Predef, proplists:get_value(link_predefined_types, Opts, false)} of + {true, false} -> []; + {true, true} -> + [{href, get_uri(N#t_name{ module = erlang }, Env)}]; + {false, _} -> + [{href, get_uri(N, Env)}] end, - {abstype, HRef, [to_xml(N, Env) | map(fun wrap_utype/2, As, Env)]}; -to_xml(#t_fun{args = As, range = T}, Env) -> - {'fun', [{argtypes, map(fun wrap_utype/2, As, Env)}, - wrap_utype(T, Env)]}; -to_xml(#t_map{ types = Ts}, Env) -> - {map, map(fun to_xml/2, Ts, Env)}; -to_xml(#t_map_field{assoc_type = AT, k_type=K, v_type=V}, Env) -> - {map_field, [{assoc_type, AT}], [wrap_utype(K,Env), wrap_utype(V, Env)]}; -to_xml(#t_tuple{types = Ts}, Env) -> - {tuple, map(fun wrap_utype/2, Ts, Env)}; -to_xml(#t_list{type = T}, Env) -> - {list, [wrap_utype(T, Env)]}; -to_xml(#t_nil{}, _Env) -> + {abstype, HRef, [to_xml(N, Env, Opts) | map(fun wrap_utype/3, As, Env, Opts)]}; +to_xml(#t_fun{args = As, range = T}, Env, Opts) -> + {'fun', [{argtypes, map(fun wrap_utype/3, As, Env, Opts)}, + wrap_utype(T, Env, Opts)]}; +to_xml(#t_map{ types = Ts}, Env, Opts) -> + {map, map(fun to_xml/3, Ts, Env, Opts)}; +to_xml(#t_map_field{assoc_type = AT, k_type=K, v_type=V}, Env, Opts) -> + {map_field, [{assoc_type, AT}], [wrap_utype(K,Env, Opts), wrap_utype(V, Env, Opts)]}; +to_xml(#t_tuple{types = Ts}, Env, Opts) -> + {tuple, map(fun wrap_utype/3, Ts, Env, Opts)}; +to_xml(#t_list{type = T}, Env, Opts) -> + {list, [wrap_utype(T, Env, Opts)]}; +to_xml(#t_nil{}, _Env, _Opts) -> nil; -to_xml(#t_paren{type = T}, Env) -> - {paren, [wrap_utype(T, Env)]}; -to_xml(#t_nonempty_list{type = T}, Env) -> - {nonempty_list, [wrap_utype(T, Env)]}; -to_xml(#t_atom{val = V}, _Env) -> +to_xml(#t_paren{type = T}, Env, Opts) -> + {paren, [wrap_utype(T, Env, Opts)]}; +to_xml(#t_nonempty_list{type = T}, Env, Opts) -> + {nonempty_list, [wrap_utype(T, Env, Opts)]}; +to_xml(#t_atom{val = V}, _Env, _Opts) -> {atom, [{value, atom_to_list(V)}], []}; -to_xml(#t_integer{val = V}, _Env) -> +to_xml(#t_integer{val = V}, _Env, _Opts) -> {integer, [{value, integer_to_list(V)}], []}; -to_xml(#t_integer_range{from = From, to = To}, _Env) -> +to_xml(#t_integer_range{from = From, to = To}, _Env, _Opts) -> {range, [{value, integer_to_list(From)++".."++integer_to_list(To)}], []}; -to_xml(#t_binary{base_size = 0, unit_size = 0}, _Ens) -> +to_xml(#t_binary{base_size = 0, unit_size = 0}, _Env, _Opts) -> {binary, [{value, "<<>>"}], []}; -to_xml(#t_binary{base_size = B, unit_size = 0}, _Ens) -> +to_xml(#t_binary{base_size = B, unit_size = 0}, _Env, _Opts) -> {binary, [{value, io_lib:fwrite("<<_:~w>>", [B])}], []}; -%to_xml(#t_binary{base_size = 0, unit_size = 8}, _Ens) -> +%to_xml(#t_binary{base_size = 0, unit_size = 8}, _Env, _Opts) -> % {binary, [{value, "binary()"}], []}; -to_xml(#t_binary{base_size = 0, unit_size = U}, _Ens) -> +to_xml(#t_binary{base_size = 0, unit_size = U}, _Env, _Opts) -> {binary, [{value, io_lib:fwrite("<<_:_*~w>>", [U])}], []}; -to_xml(#t_binary{base_size = B, unit_size = U}, _Ens) -> +to_xml(#t_binary{base_size = B, unit_size = U}, _Env, _Opts) -> {binary, [{value, io_lib:fwrite("<<_:~w, _:_*~w>>", [B, U])}], []}; -to_xml(#t_float{val = V}, _Env) -> +to_xml(#t_float{val = V}, _Env, _Opts) -> {float, [{value, io_lib:write(V)}], []}; -to_xml(#t_union{types = Ts}, Env) -> - {union, map(fun wrap_utype/2, Ts, Env)}; -to_xml(#t_record{name = N = #t_atom{}, fields = Fs}, Env) -> - {record, [to_xml(N, Env) | map(fun to_xml/2, Fs, Env)]}; -to_xml(#t_field{name = N = #t_atom{}, type = T}, Env) -> - {field, [to_xml(N, Env), wrap_type(T, Env)]}; -to_xml(#t_def{name = N = #t_var{}, type = T}, Env) -> - {localdef, [to_xml(N, Env), wrap_type(T, Env)]}; -to_xml(#t_def{name = N, type = T}, Env) -> +to_xml(#t_union{types = Ts}, Env, Opts) -> + {union, map(fun wrap_utype/3, Ts, Env, Opts)}; +to_xml(#t_record{name = N = #t_atom{}, fields = Fs}, Env, Opts) -> + {record, [to_xml(N, Env, Opts) | map(fun to_xml/3, Fs, Env, Opts)]}; +to_xml(#t_field{name = N = #t_atom{}, type = T}, Env, Opts) -> + {field, [to_xml(N, Env, Opts), wrap_type(T, Env, Opts)]}; +to_xml(#t_def{name = N = #t_var{}, type = T}, Env, Opts) -> + {localdef, [to_xml(N, Env, Opts), wrap_type(T, Env, Opts)]}; +to_xml(#t_def{name = N, type = T}, Env, Opts) -> {localdef, [{label, to_label(N)}], - [to_xml(N, Env), wrap_type(T, Env)]}; -to_xml(#t_spec{name = N, type = T, defs = Ds}, Env) -> - {typespec, [to_xml(N, Env), wrap_utype(T, Env) - | map(fun to_xml/2, Ds, Env)]}; + [to_xml(N, Env, Opts), wrap_type(T, Env, Opts)]}; +to_xml(#t_spec{name = N, type = T, defs = Ds}, Env, Opts) -> + {typespec, [to_xml(N, Env, Opts), wrap_utype(T, Env, Opts) + | map(fun to_xml/3, Ds, Env, Opts)]}; to_xml(#t_typedef{name = N, args = As, type = undefined, defs = Ds}, - Env) -> - {typedef, [to_xml(N, Env), - {argtypes, map(fun wrap_utype/2, As, Env)} - | map(fun to_xml/2, Ds, Env)]}; -to_xml(#t_typedef{name = N, args = As, type = T, defs = Ds}, Env) -> - {typedef, [to_xml(N, Env), - {argtypes, map(fun wrap_utype/2, As, Env)}, - wrap_type(T, Env) - | map(fun to_xml/2, Ds, Env)]}; -to_xml(#t_throws{type = T, defs = Ds}, Env) -> - {throws, [wrap_type(T, Env) - | map(fun to_xml/2, Ds, Env)]}. - -wrap_type(T, Env) -> - {type, [to_xml(T, Env)]}. - -wrap_utype(T, Env) -> - E = to_xml(T, Env), + Env, Opts) -> + {typedef, [to_xml(N, Env, Opts), + {argtypes, map(fun wrap_utype/3, As, Env, Opts)} + | map(fun to_xml/3, Ds, Env, Opts)]}; +to_xml(#t_typedef{name = N, args = As, type = T, defs = Ds}, Env, Opts) -> + {typedef, [to_xml(N, Env, Opts), + {argtypes, map(fun wrap_utype/3, As, Env, Opts)}, + wrap_type(T, Env, Opts) + | map(fun to_xml/3, Ds, Env, Opts)]}; +to_xml(#t_throws{type = T, defs = Ds}, Env, Opts) -> + {throws, [wrap_type(T, Env, Opts) + | map(fun to_xml/3, Ds, Env, Opts)]}. + +wrap_type(T, Env, Opts) -> + {type, [to_xml(T, Env, Opts)]}. + +wrap_utype(T, Env, Opts) -> + E = to_xml(T, Env, Opts), case arg_name(T) of '_' -> {type, [E]}; A -> {type, [{name, atom_to_list(A)}], [E]} end. -map(F, Xs, Env) -> - [F(X, Env) || X <- Xs]. +map(F, Xs, Env, Opts) -> + [F(X, Env, Opts) || X <- Xs]. is_name(A) when is_atom(A) -> true; is_name(_) -> false. diff --git a/lib/erl_docgen/priv/bin/codeline_preprocessing.escript b/lib/erl_docgen/priv/bin/codeline_preprocessing.escript index 2ee8650a18..2d84457e9a 100755 --- a/lib/erl_docgen/priv/bin/codeline_preprocessing.escript +++ b/lib/erl_docgen/priv/bin/codeline_preprocessing.escript @@ -22,7 +22,7 @@ %% %% Created : 10 Sep 2008 by Lars Thorsen %%---------------------------------------------------------------------- - +-mode(compile). %%====================================================================== %% External functions %%====================================================================== @@ -31,29 +31,24 @@ %% Description: %%---------------------------------------------------------------------- main([CPath, InFile, OutFile]) -> - InDev = + InDev = case file:open(InFile, [read]) of {ok,ID} -> ID; _ -> - halt(5) + fail(5, "Could not open ~ts for reading", [InFile]) end, - OutDev = + OutDev = case file:open(OutFile, [write]) of {ok,OD} -> OD; _ -> - halt(5) + fail(5, "Could not open ~ts for writing", [OutFile]) end, - case re:compile("<codeinclude(?:\040|\t)*file=\"([^\"]*)\"(?:(?:(?:\040|\t)*tag=\"([^\"]*)\".*)|(?:.*))(?:/>|/codeinclude>)") of - {ok, Mp} -> - parse(InDev, OutDev, CPath, Mp); - _ -> - halt(2) - end; + parse(InDev, OutDev, CPath); main(_) -> usage(). - + %%====================================================================== %% Internal functions %%====================================================================== @@ -63,61 +58,76 @@ main(_) -> %% Description: %%---------------------------------------------------------------------- usage() -> - io:format("usage: codeline_preprocessing.escript <infile> <outfile>\n"), - halt(1). + fail("usage: codeline_preprocessing.escript <infile> <outfile>\n"). %%====================================================================== %% Internal functions %%====================================================================== -parse(InDev, OutDev, CPath, Mp) -> +parse(InDev, OutDev, CPath) -> case io:get_line(InDev, "") of eof -> file:close(OutDev), file:close(InDev); - String -> - case re:run(String, Mp,[{capture, [1,2], list}]) of - {match,[File, []]} -> - case file:read_file(filename:join(CPath, File))of - {ok, Bin} -> - file:write(OutDev, "<code>\n<![CDATA[\n"), - file:write(OutDev, Bin), - file:write(OutDev, "]]></code>"); - _ -> - halt(3) - end; - {match,[File, Tag]} -> - String2 = get_code(filename:join(CPath, File), Tag), - file:write(OutDev, "<code>\n<![CDATA[\n"), - file:write(OutDev, String2), - file:write(OutDev, "]]></code>"); - _ -> - file:write(OutDev, String) - end, - parse(InDev, OutDev, CPath, Mp) + Line -> + file:write(OutDev, parse_line(CPath, Line)), + parse(InDev, OutDev, CPath) end. - + + +parse_line(CPath, Line) -> + case re:run(Line, "<codeinclude([^>]*)>.*</codeinclude>", [{capture, [1], list}]) of + {match,[Attributes]} -> + File = + case re:run(Attributes,"file=\"([^\"]+)\"",[{capture, [1], list}]) of + {match, [FileMatch]} -> + FileMatch; + nomatch -> + fail("Did not find 'file' attribute in codeinclude") + end, + FileContent = + case file:read_file(filename:join(CPath, File)) of + {ok, Bin} -> + Bin; + {error,Error} -> + fail("Could not open file ~ts (~t)~n", [File, Error]) + end, + Type = + case re:run(Attributes,"(type=\"[^\"]+\")",[{capture, [1], list}]) of + {match,[TypeMatch]} -> + TypeMatch; + nomatch -> + "" + end, + TaggedContent = + case re:run(Attributes,"tag=\"([^\"]+)\"",[{capture, [1], list}]) of + {match,[Tag]} -> + case re:run(FileContent,"^" ++ Tag ++ "\n((.|\n)*)\n" ++ + Tag ++ "\$",[global, multiline, {capture, [1], binary}]) of + {match,[[Match]]} -> + Match; + nomatch -> + fail(4, "Could not find the tag ~p in ~ts", + [Tag, File]) + end; + nomatch -> + FileContent + end, + ["<code " ++ Type ++ ">\n<![CDATA[\n", TaggedContent, "]]></code>\n"]; + nomatch -> + Line + end. + %%---------------------------------------------------------------------- %% Function: get_code/2 %% Description: %%---------------------------------------------------------------------- -get_code(File, Tag) -> - case file:read_file(File) of - {ok, Bin} -> - case re:run(Bin,"^" ++ Tag ++ "\n((.|\n)*)\n" ++ - Tag ++ "\$",[global, multiline, {capture, [1], binary}]) of - {match,[[Match]]} -> - Match; - nomatch -> - io:format(standard_error,"Could not find the tag ~p in ~ts", - [Tag, File]), - halt(4) - end; - E -> - io:format(standard_error,"Could not read ~ts (~p)",[File, E]), - halt(3) - end. - - +fail(Fmt) -> + fail(Fmt, []). +fail(Fmt, Args) -> + fail(1, Fmt, Args). +fail(Reason, Fmt, Args) -> + io:format(standard_error, Fmt, Args), + halt(Reason). diff --git a/lib/erl_docgen/priv/bin/specs_gen.escript b/lib/erl_docgen/priv/bin/specs_gen.escript index 6953664877..b6d24e4f2b 100644 --- a/lib/erl_docgen/priv/bin/specs_gen.escript +++ b/lib/erl_docgen/priv/bin/specs_gen.escript @@ -70,7 +70,7 @@ usage() -> call_edoc(FileSpec, InclFs, Dir) -> ReadOpts = [{includes, InclFs}, {preprocess, true}], - ExtractOpts = [{report_missing_type, false}], + ExtractOpts = [{report_missing_type, false}, {link_predefined_types, true}], LayoutOpts = [{pretty_printer, erl_pp}, {layout, docgen_otp_specs}], File = case FileSpec of {file, File0} -> File0; diff --git a/lib/erl_docgen/priv/dtd/erlref.dtd b/lib/erl_docgen/priv/dtd/erlref.dtd index 960a00025d..06f4198976 100644 --- a/lib/erl_docgen/priv/dtd/erlref.dtd +++ b/lib/erl_docgen/priv/dtd/erlref.dtd @@ -36,4 +36,5 @@ clause_i CDATA #IMPLIED anchor CDATA #IMPLIED since CDATA #IMPLIED - n_vars CDATA #IMPLIED> + n_vars CDATA #IMPLIED + prefix CDATA #IMPLIED> diff --git a/lib/erl_docgen/priv/xsl/db_html.xsl b/lib/erl_docgen/priv/xsl/db_html.xsl index 730c5d1447..33d6773a7b 100644 --- a/lib/erl_docgen/priv/xsl/db_html.xsl +++ b/lib/erl_docgen/priv/xsl/db_html.xsl @@ -90,7 +90,20 @@ <func:function name="erl:to-link"> <xsl:param name="text"/> - <func:result select="translate(erl:lower-case($text),'?: /()" ','--------')"/> + <xsl:variable name="link" select="translate(erl:lower-case($text),'?: /()" ','--------')"/> + <func:result> + <xsl:choose> + <!-- Stupid JS does not want us to have html elements with id="exports". + If we do, then highlight.js breaks because it uses the 'exports' + variable to do JS module things. See: + https://github.com/googlearchive/observe-js/issues/81 + for more details on this very frustrating issue... --> + <xsl:when test="$link = 'exports'">export</xsl:when> + <xsl:otherwise> + <xsl:value-of select="$link"/> + </xsl:otherwise> + </xsl:choose> + </func:result> </func:function> <!-- Used from template menu.funcs to sort a module's functions for the lefthand index list, @@ -583,7 +596,14 @@ </xsl:if> <xsl:choose> <xsl:when test="$mode = ''"> - <xsl:apply-templates select="$type/typedecl"/> + <xsl:variable name="prefix"> + <xsl:if test="@prefix = 'true'"> + <xsl:value-of select="concat($curModule,':')"/> + </xsl:if> + </xsl:variable> + <xsl:apply-templates select="$type/typedecl"> + <xsl:with-param name="prefix" select="$prefix"/> + </xsl:apply-templates> </xsl:when> <xsl:when test="$mode = 'local_type'"> <xsl:apply-templates select="$type/typedecl" mode="local_type"> @@ -600,7 +620,9 @@ </xsl:template> <xsl:template match="typehead"> + <xsl:param name="prefix"/> <span class="bold_code bc-4"> + <xsl:value-of select="$prefix"/> <xsl:apply-templates/> </span><br/> </xsl:template> @@ -2533,6 +2555,7 @@ <xsl:call-template name="title_link"> <xsl:with-param name="title" select="$title"/> <xsl:with-param name="link" select="erl:to-link($title)"/> + <xsl:with-param name="header" select="'h3'"/> </xsl:call-template> </xsl:template> diff --git a/lib/inets/doc/src/httpc.xml b/lib/inets/doc/src/httpc.xml index 905cfb2465..34010b25e4 100644 --- a/lib/inets/doc/src/httpc.xml +++ b/lib/inets/doc/src/httpc.xml @@ -4,7 +4,7 @@ <erlref> <header> <copyright> - <year>2004</year><year>2018</year> + <year>2004</year><year>2021</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -67,7 +67,7 @@ <p>Type definitions that are used more than once in this module:</p> <p><c>boolean() = true | false</c></p> - <p><c>string()</c> = list of ASCII characters</p> + <p><c>http_string()</c> = list of ASCII characters</p> <p><c>request_id() = reference()</c></p> <p><c>profile() = atom()</c></p> <p><c>path() = string()</c> representing a file path or directory path</p> @@ -90,7 +90,7 @@ <p>| <c>{url(), headers(), content_type(), body()}</c></p> </item> </taglist> - <p><c>url() = string()</c> syntax according to the URI definition in + <p><c>url() = http_string()</c> syntax according to the URI definition in <url href="http://www.ietf.org/rfc/rfc3986.txt">RFC 3986</url>, for example <c>"http://www.erlang.org"</c></p> <warning><p>Please note that httpc normalizes input URIs before internal processing @@ -102,17 +102,17 @@ creating the request: <c>httpc:request("http://localhost/foo%2525bar").</c> </p></warning> <p><c>status_line() = {http_version(), status_code(), reason_phrase()}</c></p> - <p><c>http_version() = string()</c>, for example, <c>"HTTP/1.1"</c></p> + <p><c>http_version() = http_string()</c>, for example, <c>"HTTP/1.1"</c></p> <p><c>status_code() = integer()</c></p> <p><c>reason_phrase() = string()</c></p> - <p><c>content_type() = string()</c></p> + <p><c>content_type() = http_string()</c></p> <p><c>headers() = [header()]</c></p> <p><c>header() = {field(), value()}</c></p> - <p><c>field() = string()</c></p> - <p><c>value() = string()</c></p> + <p><c>field() = [byte()]</c></p> + <p><c>value() = binary() | iolist()</c></p> <taglist> <tag><c>body()</c></tag> - <item><p>= <c>string() | binary()</c></p> + <item><p>= <c>http_string() | binary()</c></p> <p>| <c>{fun(accumulator())</c></p> <p><c> -> body_processing_result(), accumulator()}</c></p> <p>| <c>{chunkify, fun(accumulator())</c></p> @@ -269,7 +269,7 @@ <v>Url = url()</v> <v>Result = {status_line(), headers(), Body} | {status_code(), Body} | request_id()</v> - <v>Body = string() | binary()</v> + <v>Body = http_string() | binary()</v> <v>Profile = profile() | pid()</v> <d>When started <c>stand_alone</c> only the pid can be used.</d> <v>Reason = term()</v> @@ -317,7 +317,7 @@ <v>body_format() = string | binary</v> <v>Result = {status_line(), headers(), Body} | {status_code(), Body} | request_id()</v> - <v>Body = string() | binary()</v> + <v>Body = http_string() | binary()</v> <v>Profile = profile() | pid()</v> <d>When started <c>stand_alone</c> only the pid can be used.</d> <v>Reason = term()</v> @@ -549,7 +549,7 @@ <v>| {verbose, VerboseMode}</v> <v>| {unix_socket, UnixSocket}</v> <v>Proxy = {Hostname, Port}</v> - <v>Hostname = string()</v> + <v>Hostname = http_string()</v> <d>Example: "localhost" or "foo.bar.se"</d> <v>Port = integer()</v> <d>Example: 8080</d> @@ -557,7 +557,7 @@ <v>NoProxyDesc = DomainDesc | HostName | IPDesc</v> <v>DomainDesc = "*.Domain"</v> <d>Example: "*.ericsson.se"</d> - <v>IpDesc = string()</v> + <v>IpDesc = http_string()</v> <d>Example: "134.138" or "[FEDC:BA98" (all IP addresses starting with 134.138 or FEDC:BA98), "66.35.250.150" or "[2010:836B:4179::836B:4179]" (a complete IP address). diff --git a/lib/inets/src/http_client/httpc.erl b/lib/inets/src/http_client/httpc.erl index 00e960cf71..1fd65027ee 100644 --- a/lib/inets/src/http_client/httpc.erl +++ b/lib/inets/src/http_client/httpc.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2009-2018. All Rights Reserved. +%% Copyright Ericsson AB 2009-2021. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -98,8 +98,11 @@ request(Url, Profile) -> %%-------------------------------------------------------------------------- %% request(Method, Request, HTTPOptions, Options [, Profile]) -> -%% {ok, {StatusLine, Headers, Body}} | {ok, {Status, Body}} | -%% {ok, RequestId} | {error,Reason} | {ok, {saved_as, FilePath} +%% {ok, {StatusLine, Headers, Body}} +%% | {ok, {Status, Body}} +%% | {ok, RequestId} +%% | {ok, {saved_as, FilePath} +%% | {error, Reason} %% %% Method - atom() = head | get | put | patch | post | trace | %% options | delete @@ -129,10 +132,10 @@ request(Url, Profile) -> %% ReasonPhrase = string() %% Headers = [Header] %% Header = {Field, Value} -%% Field = string() -%% Value = string() +%% Field = [byte()] +%% Value = binary() | iolist() %% Body = string() | binary() | {fun(SendAcc) -> SendFunResult, SendAcc} | -%% {chunkify, fun(SendAcc) -> SendFunResult, SendAcc} - HTLM-code +%% {chunkify, fun(SendAcc) -> SendFunResult, SendAcc} - HTML-code %% SendFunResult = eof | {ok, iolist(), NewSendAcc} %% SendAcc = NewSendAcc = term() %% @@ -140,42 +143,31 @@ request(Url, Profile) -> %% syncronus and asynchronous in the later case the function will %% return {ok, RequestId} and later on a message will be sent to the %% calling process on the format {http, {RequestId, {StatusLine, -%% Headers, Body}}} or {http, {RequestId, {error, Reason}}} +%% Headers, Body}}} or {http, {RequestId, {error, Reason}}}. +%% Only octects are accepted in header fields and values. %%-------------------------------------------------------------------------- request(Method, Request, HttpOptions, Options) -> - request(Method, Request, HttpOptions, Options, default_profile()). - -request(Method, - {Url, Headers, ContentType, TupleBody}, - HTTPOptions, Options, Profile) - when ((Method =:= post) orelse (Method =:= patch) orelse (Method =:= put) orelse (Method =:= delete)) - andalso (is_atom(Profile) orelse is_pid(Profile)) andalso - is_list(ContentType) andalso is_tuple(TupleBody)-> - case check_body_gen(TupleBody) of - ok -> - do_request(Method, {Url, Headers, ContentType, TupleBody}, HTTPOptions, Options, Profile); - Error -> - Error - end; -request(Method, - {Url, Headers, ContentType, Body}, - HTTPOptions, Options, Profile) - when ((Method =:= post) orelse (Method =:= patch) orelse (Method =:= put) orelse (Method =:= delete)) - andalso (is_atom(Profile) orelse is_pid(Profile)) andalso - is_list(ContentType) andalso (is_list(Body) orelse is_binary(Body)) -> - do_request(Method, {Url, Headers, ContentType, Body}, HTTPOptions, Options, Profile); - -request(Method, - {Url, Headers}, - HTTPOptions, Options, Profile) - when (Method =:= options) orelse - (Method =:= get) orelse - (Method =:= put) orelse - (Method =:= head) orelse - (Method =:= delete) orelse - (Method =:= trace) andalso - (is_atom(Profile) orelse is_pid(Profile)) -> + request(Method, Request, HttpOptions, Options, default_profile()). + +-define(WITH_BODY, [post, put, patch, delete]). +-define(WITHOUT_BODY, [get, head, options, trace, put, delete]). + +request(Method, Request, HTTPOptions, Options, Profile) + when is_atom(Profile) orelse is_pid(Profile) -> + WithBody = lists:member(Method, ?WITH_BODY), + WithoutBody = lists:member(Method, ?WITHOUT_BODY), + case check_request(WithBody, WithoutBody, Request) of + ok -> + do_request(Method, Request, + HTTPOptions, Options, Profile); + {error, _} = Error -> + Error + end. + +do_request(Method, {Url, Headers}, HTTPOptions, Options, Profile) -> + do_request(Method, {Url, Headers, [], []}, HTTPOptions, Options, Profile); +do_request(Method, {Url, Headers, ContentType, Body}, HTTPOptions, Options, Profile) -> case normalize_and_parse_url(Url) of {error, Reason, _} -> {error, Reason}; @@ -183,21 +175,27 @@ request(Method, case header_parse(Headers) of {error, Reason} -> {error, Reason}; - _ -> - handle_request(Method, Url, ParsedUrl, Headers, [], [], - HTTPOptions, Options, Profile) - end + ok -> + handle_request(Method, Url, + ParsedUrl, Headers, ContentType, Body, + HTTPOptions, Options, Profile) + end end. -do_request(Method, {Url, Headers, ContentType, Body}, HTTPOptions, Options, Profile) -> - case normalize_and_parse_url(Url) of - {error, Reason, _} -> - {error, Reason}; - ParsedUrl -> - handle_request(Method, Url, - ParsedUrl, Headers, ContentType, Body, - HTTPOptions, Options, Profile) - end. +%% Check combination of method and presence of body +check_request(false, false, _Request) -> + {error, invalid_method}; +check_request(_, true, {_URL, _Headers}) -> + ok; +check_request(true, _, {_URL, _Headers, ContentType, Body}) + when is_list(ContentType) + andalso (is_list(Body) orelse is_binary(Body)) -> + ok; +check_request(true, _, {_URL, _Headers, ContentType, Body}) + when is_list(ContentType) andalso is_tuple(Body) -> + check_body_gen(Body); +check_request(_, _, _Request) -> + {error, invalid_request}. %%-------------------------------------------------------------------------- %% cancel_request(RequestId) -> ok @@ -1253,15 +1251,21 @@ validate_headers(RequestHeaders, _, _) -> %%-------------------------------------------------------------------------- -%% These functions is just simple wrappers to parse specifically HTTP URIs +%% These functions are just simple wrappers to parse specifically HTTP URIs %%-------------------------------------------------------------------------- header_parse([]) -> ok; -header_parse([{Field, Value}|T]) when is_list(Field), is_list(Value) -> +header_parse([{Field, Value}|T]) + when is_list(Field) + andalso (is_list(Value) orelse is_binary(Value)) -> header_parse(T); -header_parse(_) -> - {error, {headers_error, not_strings}}. +header_parse([{Field, _Value}| _ ]) + when not is_list(Field) -> + {error, {headers_error, invalid_field}}; +header_parse([{_, _}| _]) -> + {error, {headers_error, invalid_value}}. + child_name2info(undefined) -> {error, no_such_service}; child_name2info(httpc_manager) -> @@ -1277,9 +1281,9 @@ child_name(Pid, [_ | Children]) -> child_name(Pid, Children). -check_body_gen({Fun, _}) when is_function(Fun) -> +check_body_gen({Fun, _}) when is_function(Fun, 1) -> ok; -check_body_gen({chunkify, Fun, _}) when is_function(Fun) -> +check_body_gen({chunkify, Fun, _}) when is_function(Fun, 1) -> ok; -check_body_gen(Gen) -> +check_body_gen(Gen) -> {error, {bad_body_generator, Gen}}. diff --git a/lib/inets/src/http_lib/http_request.erl b/lib/inets/src/http_lib/http_request.erl index 2510cdede2..02fa8acb80 100644 --- a/lib/inets/src/http_lib/http_request.erl +++ b/lib/inets/src/http_lib/http_request.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2005-2018. All Rights Reserved. +%% Copyright Ericsson AB 2005-2021. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -201,107 +201,99 @@ headers(Key, Value, Headers) -> Headers#http_request_h{other= [{Key, Value} | Headers#http_request_h.other]}. -key_value_str(Key = 'cache-control', Headers) -> - key_value_str(atom_to_list(Key), Headers#http_request_h.'cache-control'); -key_value_str(Key = connection, Headers) -> - key_value_str(atom_to_list(Key), Headers#http_request_h.connection); -key_value_str(Key = date, Headers) -> - key_value_str(atom_to_list(Key), Headers#http_request_h.date); -key_value_str(Key = pragma, Headers) -> - key_value_str(atom_to_list(Key), Headers#http_request_h.pragma); -key_value_str(Key = trailer, Headers) -> - key_value_str(atom_to_list(Key), Headers#http_request_h.trailer); -key_value_str(Key = 'transfer-encoding', Headers) -> - key_value_str(atom_to_list(Key), - Headers#http_request_h.'transfer-encoding'); -key_value_str(Key = upgrade, Headers) -> - key_value_str(atom_to_list(Key), Headers#http_request_h.upgrade); -key_value_str(Key = via, Headers) -> - key_value_str(atom_to_list(Key), Headers#http_request_h.via); -key_value_str(Key = warning, Headers) -> - key_value_str(atom_to_list(Key), Headers#http_request_h.warning); -key_value_str(Key = accept, Headers) -> - key_value_str(atom_to_list(Key), Headers#http_request_h.accept); -key_value_str(Key = 'accept-charset', Headers) -> - key_value_str(atom_to_list(Key), Headers#http_request_h.'accept-charset'); -key_value_str(Key = 'accept-encoding', Headers) -> - key_value_str(atom_to_list(Key), Headers#http_request_h.'accept-encoding'); -key_value_str(Key = 'accept-language', Headers) -> - key_value_str(atom_to_list(Key), Headers#http_request_h.'accept-language'); -key_value_str(Key = authorization, Headers) -> - key_value_str(atom_to_list(Key), - Headers#http_request_h.authorization); -key_value_str(Key = expect, Headers) -> - key_value_str(atom_to_list(Key), Headers#http_request_h.expect); -key_value_str(Key = from, Headers) -> - key_value_str(atom_to_list(Key), Headers#http_request_h.from); -key_value_str(Key = host, Headers) -> - key_value_str(atom_to_list(Key), Headers#http_request_h.host); -key_value_str(Key = 'if-match', Headers) -> - key_value_str(atom_to_list(Key), - Headers#http_request_h.'if-match'); -key_value_str(Key = 'if-modified-since', Headers) -> - key_value_str(atom_to_list(Key), - Headers#http_request_h.'if-modified-since'); -key_value_str(Key = 'if-none-match', Headers) -> - key_value_str(atom_to_list(Key), - Headers#http_request_h.'if-none-match'); -key_value_str(Key = 'if-range', Headers) -> - key_value_str(atom_to_list(Key), - Headers#http_request_h.'if-range'); -key_value_str(Key = 'if-unmodified-since', Headers) -> - key_value_str(atom_to_list(Key), - Headers#http_request_h.'if-unmodified-since'); -key_value_str(Key = 'max-forwards', Headers) -> - key_value_str(atom_to_list(Key), - Headers#http_request_h.'max-forwards'); -key_value_str(Key = 'proxy-authorization', Headers) -> - key_value_str(atom_to_list(Key), - Headers#http_request_h.'proxy-authorization'); -key_value_str(Key = range, Headers) -> - key_value_str(atom_to_list(Key), - Headers#http_request_h.range); -key_value_str(Key = referer, Headers) -> - key_value_str(atom_to_list(Key), - Headers#http_request_h.referer); -key_value_str(Key = te, Headers) -> - key_value_str(atom_to_list(Key), - Headers#http_request_h.te); -key_value_str(Key = 'user-agent', Headers) -> - key_value_str(atom_to_list(Key), - Headers#http_request_h.'user-agent'); -key_value_str(Key = allow, Headers) -> - key_value_str(atom_to_list(Key), Headers#http_request_h.allow); -key_value_str(Key = 'content-encoding', Headers) -> - key_value_str(atom_to_list(Key), - Headers#http_request_h.'content-encoding'); -key_value_str(Key = 'content-language', Headers) -> - key_value_str(atom_to_list(Key), - Headers#http_request_h.'content-language'); -key_value_str(Key = 'content-length', Headers) -> - key_value_str(atom_to_list(Key), - Headers#http_request_h.'content-length'); -key_value_str(Key = 'content-location', Headers) -> - key_value_str(atom_to_list(Key), - Headers#http_request_h.'content-location'); -key_value_str(Key = 'content-md5', Headers) -> - key_value_str(atom_to_list(Key), - Headers#http_request_h.'content-md5'); -key_value_str(Key = 'content-range', Headers) -> - key_value_str(atom_to_list(Key), Headers#http_request_h.'content-range'); -key_value_str(Key = 'content-type', Headers) -> - key_value_str(atom_to_list(Key), Headers#http_request_h.'content-type'); -key_value_str(Key = expires, Headers) -> - key_value_str(atom_to_list(Key), Headers#http_request_h.expires); -key_value_str(Key = 'last-modified', Headers) -> - key_value_str(atom_to_list(Key), Headers#http_request_h.'last-modified'); -key_value_str(_, undefined) -> - undefined; -key_value_str(Key, Value) -> - Key ++ ": " ++ Value ++ ?CRLF. +key_value_str(Key, Headers) -> + case key_value(Key, Headers) of + undefined -> undefined; + Value -> + mk_key_value_str(atom_to_list(Key), Value) + end. + +key_value('cache-control', Headers) -> + Headers#http_request_h.'cache-control'; +key_value(connection, Headers) -> + Headers#http_request_h.connection; +key_value(date, Headers) -> + Headers#http_request_h.date; +key_value(pragma, Headers) -> + Headers#http_request_h.pragma; +key_value(trailer, Headers) -> + Headers#http_request_h.trailer; +key_value('transfer-encoding', Headers) -> + Headers#http_request_h.'transfer-encoding'; +key_value(upgrade, Headers) -> + Headers#http_request_h.upgrade; +key_value(via, Headers) -> + Headers#http_request_h.via; +key_value(warning, Headers) -> + Headers#http_request_h.warning; +key_value(accept, Headers) -> + Headers#http_request_h.accept; +key_value('accept-charset', Headers) -> + Headers#http_request_h.'accept-charset'; +key_value('accept-encoding', Headers) -> + Headers#http_request_h.'accept-encoding'; +key_value('accept-language', Headers) -> + Headers#http_request_h.'accept-language'; +key_value(authorization, Headers) -> + Headers#http_request_h.authorization; +key_value(expect, Headers) -> + Headers#http_request_h.expect; +key_value(from, Headers) -> + Headers#http_request_h.from; +key_value(host, Headers) -> + Headers#http_request_h.host; +key_value('if-match', Headers) -> + Headers#http_request_h.'if-match'; +key_value('if-modified-since', Headers) -> + Headers#http_request_h.'if-modified-since'; +key_value('if-none-match', Headers) -> + Headers#http_request_h.'if-none-match'; +key_value('if-range', Headers) -> + Headers#http_request_h.'if-range'; +key_value('if-unmodified-since', Headers) -> + Headers#http_request_h.'if-unmodified-since'; +key_value('max-forwards', Headers) -> + Headers#http_request_h.'max-forwards'; +key_value('proxy-authorization', Headers) -> + Headers#http_request_h.'proxy-authorization'; +key_value(range, Headers) -> + Headers#http_request_h.range; +key_value(referer, Headers) -> + Headers#http_request_h.referer; +key_value(te, Headers) -> + Headers#http_request_h.te; +key_value('user-agent', Headers) -> + Headers#http_request_h.'user-agent'; +key_value(allow, Headers) -> + Headers#http_request_h.allow; +key_value('content-encoding', Headers) -> + Headers#http_request_h.'content-encoding'; +key_value('content-language', Headers) -> + Headers#http_request_h.'content-language'; +key_value('content-length', Headers) -> + Headers#http_request_h.'content-length'; +key_value('content-location', Headers) -> + Headers#http_request_h.'content-location'; +key_value('content-md5', Headers) -> + Headers#http_request_h.'content-md5'; +key_value('content-range', Headers) -> + Headers#http_request_h.'content-range'; +key_value('content-type', Headers) -> + Headers#http_request_h.'content-type'; +key_value(expires, Headers) -> + Headers#http_request_h.expires; +key_value('last-modified', Headers) -> + Headers#http_request_h.'last-modified'. headers_other([], Headers) -> Headers; -headers_other([{Key,Value} | Rest], Headers) -> - Header = Key ++ ": " ++ Value ++ ?CRLF, - headers_other(Rest, [Header | Headers]). +headers_other([{Key, Value} | Rest], Headers) -> + headers_other(Rest, [mk_key_value_str(Key, Value) | Headers]). + +mk_key_value_str(Key, Value) -> + Key ++ ": " ++ value_to_list(Value) ++ ?CRLF. + +value_to_list(Binary) when is_binary(Binary) -> + binary_to_list(Binary); +value_to_list(List) when is_list(List) -> + binary_to_list(iolist_to_binary(List)). diff --git a/lib/inets/test/httpc_SUITE.erl b/lib/inets/test/httpc_SUITE.erl index b755c2ebef..f7eff5822d 100644 --- a/lib/inets/test/httpc_SUITE.erl +++ b/lib/inets/test/httpc_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2004-2018. All Rights Reserved. +%% Copyright Ericsson AB 2004-2021. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -98,6 +98,30 @@ real_requests()-> emulate_lower_versions, headers, headers_as_is, + header_type_0, + header_type_1, + header_type_2, + header_type_3, + header_type_4, + header_type_5, + header_type_6, + header_type_7, + header_type_8, + header_type_9, + header_type_10, + header_type_11, + header_type_12, + header_type_13, + header_type_14, + header_type_15, + header_type_16, + header_type_17, + header_type_18, + header_type_19, + header_type_20, + header_type_21, + header_type_22, + header_type_23, empty_body, stream, stream_to_pid, @@ -105,8 +129,11 @@ real_requests()-> stream_through_mfa, streaming_error, inet_opts, - invalid_headers, + invalid_headers_key, + invalid_headers_value, invalid_body, + invalid_body_fun, + invalid_method, no_scheme, invalid_uri, undefined_port, @@ -1217,23 +1244,106 @@ headers_conflict_chunked_with_length(Config) when is_list(Config) -> %%------------------------------------------------------------------------- -invalid_headers(Config) -> - Request = {url(group_name(Config), "/dummy.html", Config), [{"cookie", undefined}]}, - {error, _} = httpc:request(get, Request, [], []). + +invalid_headers_key(Config) -> + Request = {url(group_name(Config), "/dummy.html", Config), + [{cookie, "valid cookie"}]}, + {error, {headers_error, invalid_field}} = + httpc:request(get, Request, [], []). + +invalid_headers_value(Config) -> + Request = {url(group_name(Config), "/dummy.html", Config), + [{"cookie", atom_value}]}, + {error, {headers_error, invalid_value}} = + httpc:request(get, Request, [], []). %%------------------------------------------------------------------------- -invalid_body(Config) -> +%% Doc not generated, but we can live without that. It should be +%% [{doc, "Header type test"}]. +-define(HDR_TYPE_TEST(Name, Method, Value), + Name(Config) when is_list(Config) -> + test_header_type(Config, Method, Value)). + +?HDR_TYPE_TEST(header_type_0, get, "stringheader"). +?HDR_TYPE_TEST(header_type_1, get, <<"binary">>). +?HDR_TYPE_TEST(header_type_2, get, ["an", <<"iolist">>]). +?HDR_TYPE_TEST(header_type_3, head, "stringheader"). +?HDR_TYPE_TEST(header_type_4, head, <<"binary">>). +?HDR_TYPE_TEST(header_type_5, head, ["an", <<"iolist">>]). +?HDR_TYPE_TEST(header_type_6, post, "stringheader"). +?HDR_TYPE_TEST(header_type_7, post, <<"binary">>). +?HDR_TYPE_TEST(header_type_8, post, ["an", <<"iolist">>]). +?HDR_TYPE_TEST(header_type_9, put, "stringheader"). +?HDR_TYPE_TEST(header_type_10, put, <<"binary">>). +?HDR_TYPE_TEST(header_type_11, put, ["an", <<"iolist">>]). +?HDR_TYPE_TEST(header_type_12, delete, "stringheader"). +?HDR_TYPE_TEST(header_type_13, delete, <<"binary">>). +?HDR_TYPE_TEST(header_type_14, delete, ["an", <<"iolist">>]). +?HDR_TYPE_TEST(header_type_15, trace, "stringheader"). +?HDR_TYPE_TEST(header_type_16, trace, <<"binary">>). +?HDR_TYPE_TEST(header_type_17, trace, ["an", <<"iolist">>]). +?HDR_TYPE_TEST(header_type_18, patch, "stringheader"). +?HDR_TYPE_TEST(header_type_19, patch, <<"binary">>). +?HDR_TYPE_TEST(header_type_20, patch, ["an", <<"iolist">>]). +?HDR_TYPE_TEST(header_type_21, options, "stringheader"). +?HDR_TYPE_TEST(header_type_22, options, <<"binary">>). +?HDR_TYPE_TEST(header_type_23, options, ["an", <<"iolist">>]). + +test_header_type(Config, Method, Value) -> + {Method, Value, {ok, _Data}} = + {Method, Value, + httpc:request(Method, + make_request(Config, Method, Value), + [], + [])}. + +make_request(Config, Method, Value) -> URL = url(group_name(Config), "/dummy.html", Config), - try - httpc:request(post, {URL, [], <<"text/plain">>, "foobar"}, - [], []), - ct:fail(accepted_invalid_input) - catch - error:function_clause -> - ok + Headers = [{"other-header", Value}, + {"user-agent", Value}], + %% Generate request with or without body, depending on method. + case method_type(Method) of + read -> + {URL, Headers}; + write -> + {URL, Headers, "text/plain", ""} end. +method_type(Method) -> + case Method of + get -> read; + head -> read; + options -> read; + trace -> read; + post -> write; + put -> write; + delete -> write; + patch -> write + end. + +%%------------------------------------------------------------------------- + +invalid_body(Config) -> + URL = url(group_name(Config), "/dummy.html", Config), + {error, invalid_request} = + httpc:request(post, {URL, [], <<"text/plain">>, "foobar"}, + [], []). + +invalid_body_fun(Config) -> + URL = url(group_name(Config), "/dummy.html", Config), + BodyFun = fun() -> body_part end, + {error, {bad_body_generator, _}} = + httpc:request(post, {URL, [], "text/plain", {BodyFun, init}}, + [], []). + + +invalid_method(Config) -> + URL = url(group_name(Config), "/dummy.html", Config), + {error, invalid_method} = + httpc:request(past, {URL, [], <<"text/plain">>, "foobar"}, + [], []). + %%------------------------------------------------------------------------- binary_url(Config) -> @@ -1948,7 +2058,7 @@ setup_server_dirs(ServerRoot, DocRoot, DataDir) -> "cgi_echo" end, - inets_test_lib:copy_file(Cgi, DataDir, CgiDir), + {ok, _} = inets_test_lib:copy_file(Cgi, DataDir, CgiDir), AbsCgi = filename:join([CgiDir, Cgi]), {ok, FileInfo} = file:read_file_info(AbsCgi), ok = file:write_file_info(AbsCgi, FileInfo#file_info{mode = 8#00755}). diff --git a/lib/kernel/doc/src/eep48_chapter.xml b/lib/kernel/doc/src/eep48_chapter.xml index 2173dfd949..7d683a0ec1 100644 --- a/lib/kernel/doc/src/eep48_chapter.xml +++ b/lib/kernel/doc/src/eep48_chapter.xml @@ -172,7 +172,7 @@ as <<"1.3.0">> or <<"20.0">>.</item> <tag>edit_url := binary()</tag> - <item>a binary representing a URL to change to change the documentation itself.</item> + <item>a binary representing a URL to change the documentation itself.</item> </taglist> <p>Any key may be added to Metadata at any time. Keys that are frequently diff --git a/lib/kernel/doc/src/notes.xml b/lib/kernel/doc/src/notes.xml index fb348c06f3..1a3850131f 100644 --- a/lib/kernel/doc/src/notes.xml +++ b/lib/kernel/doc/src/notes.xml @@ -31,6 +31,48 @@ </header> <p>This document describes the changes made to the Kernel application.</p> +<section><title>Kernel 8.1.3</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p>The internal, undocumented, but used, module + <c>inet_dns</c> has been fixed to handle mDNS high bit + usage of the Class field. </p><p> Code that uses the + previously obsolete, undocumented and unused record field + <c>#dns_rr.func</c> will need to be updated since that + field is now used as a boolean flag for the mDNS high + Class bit. Code that uses the also undocumented record + <c>#dns_query</c> will need to be recompiled since a + boolean field <c>#dns_query.unicast_response</c> has been + added for the mDNS high Class bit. </p> + <p> + *** POTENTIAL INCOMPATIBILITY ***</p> + <p> + Own Id: OTP-17734 Aux Id: GH-5327, OTP-17659 </p> + </item> + <item> + <p> The fix for Linux's behaviour when reconnecting an + UDP socket in PR-5120 released in OTP-24.1.2 has been + refined to only dissolve the socket's connection before a + connect if the socket is already connected, that is: only + for a reconnect. </p><p> This allows code to open a + socket with an ephemeral port, get the port number and + connect; without the port number changing (on Linux). + This turned out to have at least one valid use case + (besides test cases). </p><p> Should one reconnect the + socket then the port number may change, on Linux; it is a + known quirk, which can be worked around by binding to a + specific port number when opening the socket. If you can + do without an ephemeral port, that is... </p> + <p> + Own Id: OTP-17736 Aux Id: GH-5279, PR-5120, OTP-17559 </p> + </item> + </list> + </section> + +</section> + <section><title>Kernel 8.1.2</title> <section><title>Fixed Bugs and Malfunctions</title> diff --git a/lib/kernel/doc/src/socket.xml b/lib/kernel/doc/src/socket.xml index d211bea310..bd744d9c15 100644 --- a/lib/kernel/doc/src/socket.xml +++ b/lib/kernel/doc/src/socket.xml @@ -1954,6 +1954,166 @@ </func> <func> + <name name="ioctl" arity="2" since="OTP @OTP-17528@"/> + <fsummary>Control device.</fsummary> + <desc> + <p>Retrieve socket (device) parameters.</p> + </desc> + </func> + + <func> + <name name="ioctl" arity="3" since="OTP @OTP-17528@"/> + <fsummary>Control device.</fsummary> + <desc> + <p>Retrieve socket (device) parameters. + This function retreives a specific parameter, + according to <c>GetRequest</c> argument. The third + argument is the (lookup) "key", identifying the + interface (usually the name of the interface).</p> + <taglist> + <tag><c>gifname</c></tag> + <item> + <p>Get the name of the interface with the specified index + (integer()). </p> + <p>Result, name of the interface, is a <c>string()</c>. </p> + </item> + + <tag><c>gifindex</c></tag> + <item> + <p>Get the index of the interface with the specified name. </p> + <p>Result, interface index, is a <c>integer()</c>. </p> + </item> + + <tag><c>gifaddr</c></tag> + <item> + <p>Get the address of the interface with the specified name. + Result, address of the interface, is a <c>socket:sockaddr()</c>. </p> + </item> + + <tag><c>gifdstaddr</c></tag> + <item> + <p>Get the destination address of the point-to-point interface + with the specified name. </p> + <p>Result, destination address of the interface, + is a <c>socket:sockaddr()</c>. </p> + </item> + + <tag><c>gifbrdaddr</c></tag> + <item> + <p>Get the droadcast address for the interface + with the specified name. </p> + <p>Result, broadcast address of the interface, + is a <c>socket:sockaddr()</c>. </p> + </item> + + <tag><c>gifnetmask</c></tag> + <item> + <p>Get the network mask for the interface + with the specified name. </p> + <p>Result, network mask of the interface, + is a <c>socket:sockaddr()</c>. </p> + </item> + + <tag><c>gifhwaddr</c></tag> + <item> + <p>Get the hardware address for the interface + with the specified name. </p> + <p>Result, hardware address of the interface, + is a <c>socket:sockaddr()</c>. The family field + contains the 'ARPHRD' device type (or an integer). </p> + </item> + + <tag><c>gifmtu</c></tag> + <item> + <p>Get the MTU (Maximum Transfer Unit) for the interface + with the specified name. </p> + <p>Result, MTU of the interface, is an <c>integer()</c>. </p> + </item> + + <tag><c>giftxqlen</c></tag> + <item> + <p>Get the transmit queue length of the interface + with the specified name. </p> + <p>Result, transmit queue length of the interface, + is an <c>integer()</c>. </p> + </item> + + <tag><c>gifflags</c></tag> + <item> + <p>Get the active flag word of the interface + with the specified name. </p> + <p>Result, the active flag word of the interface, + is an list of <c>socket:ioctl_device_flag() | integer()</c>. </p> + </item> + </taglist> + + </desc> + </func> + + <func> + <name name="ioctl" arity="4" since="OTP @OTP-17528@"/> + <fsummary>Control device.</fsummary> + <desc> + <p>Set socket (device) parameters. + This function sets a specific parameter, + according to <c>SetRequest</c> argument. The third + argument is the "key", identifying the + interface (usually the name of the interface), and the + fourth is the "new" value.</p> + <p>These are privileged operation's. </p> + <taglist> + + <tag><c>sifflags</c></tag> + <item> + <p>Set the the active flag word, <c>#{Flag => boolean()}</c>, + of the interface with the specified name. </p> + <p>Each flag to be changed, should be added to the value map, + with the value <c>'true'</c> if the flag (<c>Flag</c>) + should be set and <c>'false'</c> if the flag should be reset. </p> + </item> + + <tag><c>sifaddr</c></tag> + <item> + <p>Set the address, <c>sockaddr()</c>, + of the interface with the specified name. </p> + </item> + + <tag><c>sifdstaddr</c></tag> + <item> + <p>Set the destination address, <c>sockaddr()</c>, + of a point-to-point interface with the specified name. </p> + </item> + + <tag><c>sifbrdaddr</c></tag> + <item> + <p>Set the broadcast address, <c>sockaddr()</c>, + of the interface with the specified name. </p> + </item> + + <tag><c>sifnetmask</c></tag> + <item> + <p>Set the network mask, <c>sockaddr()</c>, + of the interface with the specified name. </p> + </item> + + <tag><c>sifmtu</c></tag> + <item> + <p>Set the MTU (Maximum Transfer Unit), <c>integer()</c>, + for the interface with the specified name. </p> + </item> + + <tag><c>siftxqlen</c></tag> + <item> + <p>Set the transmit queue length, <c>integer()</c>, + of the interface with the specified name. </p> + </item> + + </taglist> + + </desc> + </func> + + <func> <name name="is_supported" arity="1" since="OTP 23.0"/> <name name="is_supported" arity="2" since="OTP 23.0"/> <fsummary>Report info about what the platform supports.</fsummary> diff --git a/lib/kernel/examples/erl_uds_dist/src/erl_uds_dist.erl b/lib/kernel/examples/erl_uds_dist/src/erl_uds_dist.erl index 44035e8e42..c69947184f 100644 --- a/lib/kernel/examples/erl_uds_dist/src/erl_uds_dist.erl +++ b/lib/kernel/examples/erl_uds_dist/src/erl_uds_dist.erl @@ -755,8 +755,10 @@ close(ListeningSocket) -> %% Get the listening socket address in a {local, Pathname} format {ok, SocketAddress} = inet:sockname(ListeningSocket), {local, SocketPathname} = SocketAddress, - %% Remove the socket file from the filesystem - file:delete(SocketPathname), + %% Remove the socket file from the filesystem. The raw option is used + %% to bypass the need for a file server, which may not be available + %% and registered anymore (for instance during a node shutdown phase). + file:delete(SocketPathname, [raw]), gen_tcp:close(ListeningSocket). diff --git a/lib/kernel/src/dist_util.erl b/lib/kernel/src/dist_util.erl index 5930300747..ae00d885fc 100644 --- a/lib/kernel/src/dist_util.erl +++ b/lib/kernel/src/dist_util.erl @@ -828,7 +828,9 @@ split_node(Node) -> Split; false -> {host,Host} - end + end; + _ -> + {error, {invalid_node, Node}} end. %% Check if connecting node is allowed to connect diff --git a/lib/kernel/src/erl_erts_errors.erl b/lib/kernel/src/erl_erts_errors.erl index 8b0768ac80..b6a8ce846f 100644 --- a/lib/kernel/src/erl_erts_errors.erl +++ b/lib/kernel/src/erl_erts_errors.erl @@ -589,12 +589,19 @@ format_erlang_error(node, [_], _) -> [not_pid]; format_erlang_error(nodes, [_], _) -> [<<"not a valid node type">>]; -format_erlang_error(open_port, [Name,_Settings], Cause) -> +format_erlang_error(open_port, [Name, Settings], Cause) -> case Cause of badopt -> [must_be_tuple(Name),bad_option]; + _ when is_tuple(Name) -> + case lists:keysearch(args, 1, Settings) of + {value,_} when element(1,Name) =/= spawn_executable -> + [<<"must be spawn_executable">>]; + _ -> + [<<"invalid port name">>] + end; _ -> - [must_be_tuple(Name, <<"invalid port name">>)] + must_be_tuple(Name) end; format_erlang_error(phash, [_,N], _) -> [must_be_pos_int(N)]; diff --git a/lib/kernel/src/erts_debug.erl b/lib/kernel/src/erts_debug.erl index b14dcadd4d..06d91d5faf 100644 --- a/lib/kernel/src/erts_debug.erl +++ b/lib/kernel/src/erts_debug.erl @@ -35,7 +35,8 @@ flat_size/1, get_internal_state/1, instructions/0, interpreter_size/0, map_info/1, same/2, set_internal_state/2, - size_shared/1, copy_shared/1, dirty_cpu/2, dirty_io/2, dirty/3, + size_shared/1, copy_shared/1, copy_shared/2, + dirty_cpu/2, dirty_io/2, dirty/3, lcnt_control/1, lcnt_control/2, lcnt_collect/0, lcnt_clear/0, lc_graph/0, lc_graph_to_dot/2, lc_graph_merge/2, alloc_blocks_size/1]). @@ -96,7 +97,14 @@ size_shared(_) -> -spec copy_shared(Term) -> term() when Term :: term(). -copy_shared(_) -> +copy_shared(Term) -> + copy_shared(Term, false). + +-spec copy_shared(Term, CopyLiterals) -> term() when + Term :: term(), + CopyLiterals :: true | false. + +copy_shared(_, _) -> erlang:nif_error(undef). -spec get_internal_state(W) -> term() when diff --git a/lib/kernel/src/gen_tcp_socket.erl b/lib/kernel/src/gen_tcp_socket.erl index 339dadad3f..7175e13836 100644 --- a/lib/kernel/src/gen_tcp_socket.erl +++ b/lib/kernel/src/gen_tcp_socket.erl @@ -33,6 +33,7 @@ monitor/1, cancel_monitor/1, setopts/2, getopts/2, sockname/1, peername/1, + socknames/1, getstat/2 ]). @@ -198,7 +199,9 @@ extra_opts(OpenOpts) when is_list(OpenOpts) -> maps:from_list(OpenOpts). -default_any(Domain, undefined = Undefined) -> +default_any(_Domain, undefined, #{fd := _}) -> + undefined; +default_any(Domain, undefined, _Opts) -> if Domain =:= inet; Domain =:= inet6 -> @@ -206,9 +209,9 @@ default_any(Domain, undefined = Undefined) -> addr => any, port => 0}; true -> - Undefined + undefined end; -default_any(_Domain, BindAddr) -> +default_any(_Domain, BindAddr, _Opts) -> BindAddr. bind_addr(_Domain, BindIP, BindPort) @@ -290,7 +293,10 @@ listen_open(Domain, ListenOpts, StartOpts, ExtraOpts, Backlog, BindAddr) -> [{start_opts, StartOpts} | setopts_opts(ErrRef, ListenOpts)]), ok(ErrRef, call(Server, {setopts, Setopts})), - ok(ErrRef, call_bind(Server, default_any(Domain, BindAddr))), + ok(ErrRef, call_bind( + Server, + default_any(Domain, BindAddr, ExtraOpts) + )), Socket = val(ErrRef, call(Server, {listen, Backlog})), {ok, ?MODULE_socket(Server, Socket)} catch @@ -519,11 +525,13 @@ setopts(?MODULE_socket(Server, _Socket), Opts) when is_list(Opts) -> call(Server, {setopts, normalize_setopts(Opts)}). + %% ------------------------------------------------------------------------- getopts(?MODULE_socket(Server, _Socket), Opts) when is_list(Opts) -> call(Server, {getopts, normalize_getopts(Opts)}). + %% ------------------------------------------------------------------------- sockname(?MODULE_socket(_Server, Socket)) -> @@ -532,6 +540,16 @@ sockname(?MODULE_socket(_Server, Socket)) -> {error, _} = Error -> Error end. + +%% ------------------------------------------------------------------------- + +socknames(Socket) -> + case sockname(Socket) of + {ok, Addr} -> {ok, [Addr]}; + {error, _} = Error -> Error + end. + + %% ------------------------------------------------------------------------- peername(?MODULE_socket(_Server, Socket)) -> diff --git a/lib/kernel/src/gen_udp_socket.erl b/lib/kernel/src/gen_udp_socket.erl index 0eb15de202..505dfea4ef 100644 --- a/lib/kernel/src/gen_udp_socket.erl +++ b/lib/kernel/src/gen_udp_socket.erl @@ -38,6 +38,7 @@ monitor/1, cancel_monitor/1, setopts/2, getopts/2, sockname/1, peername/1, + socknames/1, getstat/2 ]). @@ -115,14 +116,21 @@ close_server(Server) -> %% -- connect ---------------------------------------------------------------- connect(?MODULE_socket(_Server, Socket), Address, Port) -> + Dest = dest2sockaddr({Address, Port}), case os:type() of {unix,linux} -> - _ = socket:connect(Socket, #{family => unspec}), - ok; + case socket:peername(Socket) of + {error, enotconn} -> + socket:connect(Socket, Dest); + {error, closed} = Error -> + Error; + _ -> % Matches {ok, _} and unknown errors + _ = socket:connect(Socket, #{family => unspec}), + socket:connect(Socket, Dest) + end; _ -> - ok - end, - socket:connect(Socket, dest2sockaddr({Address, Port})). + socket:connect(Socket, Dest) + end. %% -- open ----------------------------------------------------------------- @@ -523,6 +531,15 @@ sockname(?MODULE_socket(_Server, Socket)) -> %% ------------------------------------------------------------------------- +socknames(Socket) -> + case sockname(Socket) of + {ok, Addr} -> {ok, [Addr]}; + {error, _} = Error -> Error + end. + + +%% ------------------------------------------------------------------------- + peername(?MODULE_socket(_Server, Socket)) -> case socket:peername(Socket) of {ok, SockAddr} -> {ok, address(SockAddr)}; diff --git a/lib/kernel/src/inet.erl b/lib/kernel/src/inet.erl index 36f79eed82..99bc8a6278 100644 --- a/lib/kernel/src/inet.erl +++ b/lib/kernel/src/inet.erl @@ -335,6 +335,8 @@ setsockname(Socket, undefined) -> returned_non_ip_address()]} | {error, posix()}. +socknames(?module_socket(GenSocketMod, _) = Socket) when is_atom(GenSocketMod) -> + GenSocketMod:?FUNCTION_NAME(Socket); socknames(Socket) -> prim_inet:socknames(Socket). diff --git a/lib/kernel/src/inet_dns.erl b/lib/kernel/src/inet_dns.erl index b224aa22dc..c236dfd0d6 100644 --- a/lib/kernel/src/inet_dns.erl +++ b/lib/kernel/src/inet_dns.erl @@ -28,6 +28,7 @@ %% RFC 2915: The Naming Authority Pointer (NAPTR) DNS Resource Rec %% RFC 6488: DNS Certification Authority Authorization (CAA) Resource Record %% RFC 7553: The Uniform Resource Identifier (URI) DNS Resource Record +%% RFC 6762: Multicast DNS -export([decode/1, encode/1]). @@ -199,13 +200,16 @@ decode_query_section(Rest, 0, _Buffer, Qs) -> decode_query_section(Bin, N, Buffer, Qs) -> ?MATCH_ELSE_DECODE_ERROR( decode_name(Bin, Buffer), - {<<Type:16,Class:16,Rest/binary>>,Name}, + {<<T:16,C:16,Rest/binary>>,Name}, begin - DnsQuery = - #dns_query{domain=Name, - type=decode_type(Type), - class=decode_class(Class)}, - decode_query_section(Rest, N-1, Buffer, [DnsQuery|Qs]) + {Class,UnicastResponse} = decode_class(C), + DnsQuery = + #dns_query{ + domain = Name, + type = decode_type(T), + class = Class, + unicast_response = UnicastResponse}, + decode_query_section(Rest, N-1, Buffer, [DnsQuery|Qs]) end). decode_rr_section(Bin, N, Buffer) -> @@ -223,27 +227,29 @@ decode_rr_section(Bin, N, Buffer, RRs) -> Name}, begin Type = decode_type(T), - Class = decode_class(C), - Data = decode_data(D, Class, Type, Buffer), RR = case Type of - opt -> + ?S_OPT -> <<ExtRcode,Version,Z:16>> = TTL, - #dns_rr_opt{domain=Name, - type=Type, - udp_payload_size=C, - ext_rcode=ExtRcode, - version=Version, - z=Z, - data=Data}; + #dns_rr_opt{ + domain = Name, + type = Type, + udp_payload_size = C, + ext_rcode = ExtRcode, + version = Version, + z = Z, + data = D}; _ -> + {Class,CacheFlush} = decode_class(C), + Data = decode_data(D, Class, Type, Buffer), <<TimeToLive:32/signed>> = TTL, - #dns_rr{domain=Name, - type=Type, - class=Class, - ttl=if TimeToLive < 0 -> 0; - true -> TimeToLive end, - data=Data} + #dns_rr{ + domain = Name, + type = Type, + class = Class, + ttl = max(0, TimeToLive), + data = Data, + func = CacheFlush} end, decode_rr_section(Rest, N-1, Buffer, [RR|RRs]) end). @@ -286,41 +292,51 @@ encode_header(#dns_header{id=Id}=H, QdCount, AnCount, NsCount, ArCount) -> %% encode_query_section(Bin, Comp, []) -> {Bin,Comp}; encode_query_section(Bin0, Comp0, [#dns_query{domain=DName}=Q | Qs]) -> - Type = encode_type(Q#dns_query.type), - Class = encode_class(Q#dns_query.class), + T = encode_type(Q#dns_query.type), + C = encode_class(Q#dns_query.class, Q#dns_query.unicast_response), {Bin,Comp} = encode_name(Bin0, Comp0, byte_size(Bin0), DName), - encode_query_section(<<Bin/binary,Type:16,Class:16>>, Comp, Qs). + encode_query_section(<<Bin/binary,T:16,C:16>>, Comp, Qs). %% RFC 1035: 4.1.3. Resource record format %% RFC 2671: 4.3, 4.4, 4.6 OPT RR format %% encode_res_section(Bin, Comp, []) -> {Bin,Comp}; -encode_res_section(Bin, Comp, [#dns_rr {domain = DName, - type = Type, - class = Class, - ttl = TTL, - data = Data} | Rs]) -> - encode_res_section_rr(Bin, Comp, Rs, - DName, Type, Class, <<TTL:32/signed>>, Data); -encode_res_section(Bin, Comp, [#dns_rr_opt {domain = DName, - udp_payload_size = UdpPayloadSize, - ext_rcode = ExtRCode, - version = Version, - z = Z, - data = Data} | Rs]) -> - encode_res_section_rr(Bin, Comp, Rs, - DName, ?S_OPT, UdpPayloadSize, - <<ExtRCode,Version,Z:16>>, Data). - -encode_res_section_rr(Bin0, Comp0, Rs, DName, Type, Class, TTL, Data) -> +encode_res_section( + Bin, Comp, + [#dns_rr{ + domain = DName, + type = Type, + class = Class, + func = CacheFlush, + ttl = TTL, + data = Data} | Rs]) -> + encode_res_section_rr( + Bin, Comp, Rs, DName, Type, Class, CacheFlush, + <<TTL:32/signed>>, Data); +encode_res_section( + Bin, Comp, + [#dns_rr_opt{ + domain = DName, + udp_payload_size = UdpPayloadSize, + ext_rcode = ExtRCode, + version = Version, + z = Z, + data = Data} | Rs]) -> + encode_res_section_rr( + Bin, Comp, Rs, DName, ?S_OPT, UdpPayloadSize, false, + <<ExtRCode,Version,Z:16>>, Data). + +encode_res_section_rr( + Bin0, Comp0, Rs, DName, Type, Class, CacheFlush, TTL, Data) -> T = encode_type(Type), - C = encode_class(Class), + C = encode_class(Class, CacheFlush), {Bin,Comp1} = encode_name(Bin0, Comp0, byte_size(Bin0), DName), - {DataBin,Comp} = encode_data(Comp1, byte_size(Bin)+2+2+byte_size(TTL)+2, - Type, Class, Data), + Pos = byte_size(Bin)+2+2+byte_size(TTL)+2, + {DataBin,Comp} = encode_data(Comp1, Pos, Type, Class, Data), DataSize = byte_size(DataBin), - encode_res_section(<<Bin/binary,T:16,C:16, - TTL/binary,DataSize:16,DataBin/binary>>, Comp, Rs). + encode_res_section( + <<Bin/binary,T:16,C:16,TTL/binary,DataSize:16,DataBin/binary>>, + Comp, Rs). %% %% Resource types @@ -404,25 +420,39 @@ encode_type(Type) -> Type when is_integer(Type) -> Type %% raw unknown type end. + %% -%% Resource clases +%% Resource classes %% -decode_class(Class) -> - case Class of - ?C_IN -> in; - ?C_CHAOS -> chaos; - ?C_HS -> hs; - ?C_ANY -> any; - _ -> Class %% raw unknown class - end. +decode_class(C0) -> + FlagBit = 16#8000, + C = C0 band (bnot FlagBit), + Class = + case C of + ?C_IN -> in; + ?C_CHAOS -> chaos; + ?C_HS -> hs; + ?C_ANY -> any; + _ -> C %% raw unknown class + end, + Flag = (C0 band FlagBit) =/= 0, + {Class,Flag}. + +encode_class(Class, Flag) -> + C = encode_class(Class), + case Flag of + true -> FlagBit = 16#8000, C bor FlagBit; + false -> C + end. +%% encode_class(Class) -> case Class of - in -> ?C_IN; + in -> ?C_IN; chaos -> ?C_CHAOS; - hs -> ?C_HS; - any -> ?C_ANY; + hs -> ?C_HS; + any -> ?C_ANY; Class when is_integer(Class) -> Class %% raw unknown class end. @@ -455,7 +485,7 @@ decode_boolean(I) when is_integer(I) -> true. %% Data field -> term() content representation %% %% Class IN RRs -decode_data(Data, in, ?S_A, _) -> +decode_data(Data, in, ?S_A, _) -> ?MATCH_ELSE_DECODE_ERROR(Data, <<A,B,C,D>>, {A,B,C,D}); decode_data(Data, in, ?S_AAAA, _) -> ?MATCH_ELSE_DECODE_ERROR( @@ -467,15 +497,12 @@ decode_data(Data, in, ?S_WKS, _) -> Data, <<A,B,C,D,Proto,BitMap/binary>>, {{A,B,C,D},Proto,BitMap}); -%% OPT pseudo-RR (of no class) -decode_data(Data, _UdpPayloadSize, ?S_OPT, _) -> - Data; %% decode_data(Data, Class, Type, Buffer) -> if - is_integer(Class) -> + is_integer(Class) -> % Raw class Data; - is_atom(Class) -> + is_atom(Class) -> % Symbolic class, i.e: known decode_data(Data, Type, Buffer) end. %% @@ -665,7 +692,9 @@ encode_data(Comp, _, ?S_WKS, in, Data) -> {{A,B,C,D},Proto,BitMap} = Data, BitMapBin = iolist_to_binary(BitMap), {<<A,B,C,D,Proto,BitMapBin/binary>>,Comp}; -%% OPT pseudo-RR (of no class) +%% OPT pseudo-RR (of no class) - should not take this way; +%% this must be a #dns_rr{type = ?S_OPT} instead of a #dns_rr_opt{}, +%% so good luck getting in particular Class and TTL right... encode_data(Comp, _, ?S_OPT, _UdpPayloadSize, Data) -> encode_data(Comp, Data); %% @@ -673,7 +702,7 @@ encode_data(Comp, Pos, Type, Class, Data) -> if is_integer(Class) -> encode_data(Comp, Data); - is_atom(Class) -> + is_atom(Class) -> % Known Class encode_data(Comp, Pos, Type, Data) end. %% diff --git a/lib/kernel/src/inet_dns.hrl b/lib/kernel/src/inet_dns.hrl index 04ccb15b93..c87786390c 100644 --- a/lib/kernel/src/inet_dns.hrl +++ b/lib/kernel/src/inet_dns.hrl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1997-2020. All Rights Reserved. +%% Copyright Ericsson AB 1997-2021. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -193,7 +193,8 @@ %% tm, %% creation time bm = [], %% Bitmap storing domain character case information. - func = false %% Optional function calculating the data field. + func = false %% Was: Optional function calculating the data field. + %% Now: cache-flush Class flag from mDNS RFC 6762 }). -define(DNS_UDP_PAYLOAD_SIZE, 1280). @@ -211,7 +212,8 @@ -record(dns_query, { - domain, %% query domain - type, %% query type - class %% query class + domain, %% query domain + type, %% query type + class, %% query class + unicast_response = false %% mDNS RFC 6762 Class flag }). diff --git a/lib/kernel/src/inet_tcp_dist.erl b/lib/kernel/src/inet_tcp_dist.erl index ef0f06aef7..99d8ddaba8 100644 --- a/lib/kernel/src/inet_tcp_dist.erl +++ b/lib/kernel/src/inet_tcp_dist.erl @@ -39,9 +39,6 @@ -include("net_address.hrl"). - - - -include("dist.hrl"). -include("dist_util.hrl"). @@ -54,10 +51,13 @@ select(Node) -> gen_select(inet_tcp, Node). gen_select(Driver, Node) -> - case split_node(atom_to_list(Node), $@, []) of - [_, Host] -> - case inet:getaddr(Host, Driver:family()) of - {ok,_} -> true; + case dist_util:split_node(Node) of + {node, Name, Host} -> + case call_epmd_function( + net_kernel:epmd_module(), address_please, + [Name, Host, Driver:family()]) of + {ok, _Addr} -> true; + {ok, _Addr, _Port, _Creation} -> true; _ -> false end; _ -> false diff --git a/lib/kernel/src/kernel.app.src b/lib/kernel/src/kernel.app.src index 4b35dd1f46..9a0639e90c 100644 --- a/lib/kernel/src/kernel.app.src +++ b/lib/kernel/src/kernel.app.src @@ -155,6 +155,6 @@ {shell_docs_ansi,auto} ]}, {mod, {kernel, []}}, - {runtime_dependencies, ["erts-12.0", "stdlib-3.13", "sasl-3.0", "crypto-5.0"]} + {runtime_dependencies, ["erts-@OTP-17714@", "stdlib-3.13", "sasl-3.0", "crypto-5.0"]} ] }. diff --git a/lib/kernel/src/kernel.appup.src b/lib/kernel/src/kernel.appup.src index d05a2bb07e..801293765f 100644 --- a/lib/kernel/src/kernel.appup.src +++ b/lib/kernel/src/kernel.appup.src @@ -51,7 +51,8 @@ {<<"^8\\.0\\.2(?:\\.[0-9]+)*$">>,[restart_new_emulator]}, {<<"^8\\.1$">>,[restart_new_emulator]}, {<<"^8\\.1\\.0(?:\\.[0-9]+)+$">>,[restart_new_emulator]}, - {<<"^8\\.1\\.1(?:\\.[0-9]+)*$">>,[restart_new_emulator]}], + {<<"^8\\.1\\.1(?:\\.[0-9]+)*$">>,[restart_new_emulator]}, + {<<"^8\\.1\\.2(?:\\.[0-9]+)*$">>,[restart_new_emulator]}], [{<<"^6\\.4$">>,[restart_new_emulator]}, {<<"^6\\.4\\.0(?:\\.[0-9]+)+$">>,[restart_new_emulator]}, {<<"^6\\.4\\.1(?:\\.[0-9]+)*$">>,[restart_new_emulator]}, @@ -75,4 +76,5 @@ {<<"^8\\.0\\.2(?:\\.[0-9]+)*$">>,[restart_new_emulator]}, {<<"^8\\.1$">>,[restart_new_emulator]}, {<<"^8\\.1\\.0(?:\\.[0-9]+)+$">>,[restart_new_emulator]}, - {<<"^8\\.1\\.1(?:\\.[0-9]+)*$">>,[restart_new_emulator]}]}. + {<<"^8\\.1\\.1(?:\\.[0-9]+)*$">>,[restart_new_emulator]}, + {<<"^8\\.1\\.2(?:\\.[0-9]+)*$">>,[restart_new_emulator]}]}. diff --git a/lib/kernel/src/logger.erl b/lib/kernel/src/logger.erl index 9107bf0e61..31995cfa25 100644 --- a/lib/kernel/src/logger.erl +++ b/lib/kernel/src/logger.erl @@ -988,11 +988,19 @@ get_logger_level() -> end. get_primary_metadata() -> - case application:get_env(kernel,logger_default_metadata,#{}) of - Meta when is_map(Meta) -> + case application:get_env(kernel,logger_metadata) of + {ok, Meta} when is_map(Meta) -> Meta; - Meta -> - throw({logger_metadata, Meta}) + {ok, Meta} -> + throw({logger_metadata, Meta}); + undefined -> + %% This case is here to keep bug compatability. Can be removed in OTP 25. + case application:get_env(kernel,logger_default_metadata,#{}) of + Meta when is_map(Meta) -> + Meta; + Meta -> + throw({logger_metadata, Meta}) + end end. get_primary_filter_default(Env) -> diff --git a/lib/kernel/src/socket.erl b/lib/kernel/src/socket.erl index b92de6ba07..c188f516a1 100644 --- a/lib/kernel/src/socket.erl +++ b/lib/kernel/src/socket.erl @@ -69,6 +69,8 @@ sockname/1, peername/1, + ioctl/2, ioctl/3, ioctl/4, + cancel/2 ]). @@ -141,6 +143,9 @@ info_keys/0 ]). +%% DUMMY +-export_type([ioctl_device_flag/0, ioctl_device_map/0]). + %% We need #file_descriptor{} for sendfile/2,3,4,5 -include("file_int.hrl"). @@ -718,6 +723,31 @@ ]. +%% Note that not all flags exist on all platforms! +-type ioctl_device_flag() :: up | broadcast | debug | loopback | pointopoint | + notrailers | knowsepoch | running | noarp | promisc | allmulti | + master | oactive | slave | simplex | + link0 | link1 | link2 | + multicast | portsel | automedia | + cantconfig | ppromisc | + dynamic | + monitor | staticarp | dying | renaming | nogroup | + lower_up | dormant | echo. + +%% When reading the device map (gifmap), the resulting map will be +%% "fully" populated. +%% <DOES-THIS-WORK> +%% When writing, it is expected that only the fields that is +%% to be set is present. +%% </DOES-THIS-WORK> +-type ioctl_device_map() :: #{mem_start := non_neg_integer(), + mem_end := non_neg_integer(), + base_addr := non_neg_integer(), + irq := non_neg_integer(), + dma := non_neg_integer(), + port := non_neg_integer()}. + + %% =========================================================================== %% %% Interface term formats @@ -809,9 +839,9 @@ which_sockets(Proto) Proto =:= udp -> ?REGISTRY:which_sockets({protocol, Proto}); -which_sockets(CTRL) - when is_pid(CTRL) -> - ?REGISTRY:which_sockets({ctrl, CTRL}); +which_sockets(Owner) + when is_pid(Owner) -> + ?REGISTRY:which_sockets({owner, Owner}); which_sockets(Filter) when is_function(Filter, 1) -> ?REGISTRY:which_sockets(Filter); @@ -1202,22 +1232,14 @@ info(Socket) -> -spec monitor(Socket) -> reference() when Socket :: socket(). -%% Should it be possible to specify a modification of the 'Socket' part -%% of the DOWN-message? The point would be to make it possible for -%% a gen_tcp_socket-socket to use this and get the proper 'socket' -%% as part of the message. - monitor(?socket(SockRef) = Socket) when is_reference(SockRef) -> case prim_socket:setopt(SockRef, {otp, use_registry}, true) of ok -> - case socket_registry:monitor(Socket) of - {error, MReason} -> - erlang:error({invalid, MReason}); - MRef when is_reference(MRef) -> - MRef - end; - {error, SReason} -> - erlang:error({invalid, SReason}) + socket_registry:monitor(Socket); + {error, closed = SReason} -> + MRef = make_ref(), + self() ! {'DOWN', MRef, socket, Socket, SReason}, + MRef end; monitor(Socket) -> erlang:error(badarg, [Socket]). @@ -1268,7 +1290,8 @@ cancel_monitor(MRef) -> boolean()}]}]}]. supports() -> [{Key1, supports(Key1)} - || Key1 <- [options, msg_flags, protocols]] + || Key1 <- [ioctl_requests, ioctl_flags, + options, msg_flags, protocols]] ++ prim_socket:supports(). -spec supports(Key1 :: term()) -> @@ -4028,6 +4051,174 @@ peername(Socket) -> erlang:error(badarg, [Socket]). + +%% =========================================================================== +%% +%% ioctl - control device - get requests +%% +%% + +-spec ioctl(Socket, GetRequest) -> {'ok', IFConf} | {'error', Reason} when + Socket :: socket(), + GetRequest :: 'gifconf', + IFConf :: [#{name := string, addr := sockaddr()}], + Reason :: posix() | 'closed'. + +%% gifconf | {gifaddr, string()} | {gifindex, string()} | {gifname, integer()} +ioctl(?socket(SockRef), gifconf = GetRequest) -> + prim_socket:ioctl(SockRef, GetRequest); +ioctl(Socket, GetRequest) -> + erlang:error(badarg, [Socket, GetRequest]). + +%% -spec ioctl(Socket, GetRequest, Index) -> {'ok', Name} | {'error', Reason} when +%% Socket :: socket(), +%% GetRequest :: 'gifname', +%% Index :: integer(), +%% Name :: string(), +%% Reason :: posix() | 'closed'; +%% (Socket, GetRequest, Name) -> {'ok', Index} | {'error', Reason} when +%% Socket :: socket(), +%% GetRequest :: 'gifindex', +%% Name :: string(), +%% Index :: integer(), +%% Reason :: posix() | 'closed'; +%% (Socket, GetRequest, Name) -> {'ok', Addr} | {'error', Reason} when +%% Socket :: socket(), +%% GetRequest :: 'gifaddr', +%% Name :: string(), +%% Addr :: sockaddr(), +%% Reason :: posix() | 'closed'; +%% (Socket, GetRequest, Name) -> {'ok', DestAddr} | {'error', Reason} when +%% Socket :: socket(), +%% GetRequest :: 'gifdstaddr', +%% Name :: string(), +%% DestAddr :: sockaddr(), +%% Reason :: posix() | 'closed'; +%% (Socket, GetRequest, Name) -> {'ok', BroadcastAddr} | {'error', Reason} when +%% Socket :: socket(), +%% GetRequest :: 'gifbrdaddr', +%% Name :: string(), +%% BroadcastAddr :: sockaddr(), +%% Reason :: posix() | 'closed'; +%% (Socket, GetRequest, Name) -> {'ok', Netmask} | {'error', Reason} when +%% Socket :: socket(), +%% GetRequest :: 'gifnetmask', +%% Name :: string(), +%% Netmask :: sockaddr(), +%% Reason :: posix() | 'closed'; +%% (Socket, GetRequest, Name) -> {'ok', HWAddr} | {'error', Reason} when +%% Socket :: socket(), +%% GetRequest :: 'gifhwaddr', +%% Name :: string(), +%% HWAddr :: sockaddr(), +%% Reason :: posix() | 'closed'; +%% (Socket, GetRequest, Name) -> {'ok', MTU} | {'error', Reason} when +%% Socket :: socket(), +%% GetRequest :: 'gifmtu', +%% Name :: string(), +%% MTU :: integer(), +%% Reason :: posix() | 'closed'; +%% (Socket, GetRequest, Name) -> {'ok', TransmitQLen} | {'error', Reason} when +%% Socket :: socket(), +%% GetRequest :: 'giftxqlen', +%% Name :: string(), +%% TransmitQLen :: integer(), +%% Reason :: posix() | 'closed'; +%% (Socket, GetRequest, Name) -> {'ok', Flags} | {'error', Reason} when +%% Socket :: socket(), +%% GetRequest :: 'gifflags', +%% Name :: string(), +%% Flags :: [ioctl_device_flag() | integer()], +%% Reason :: posix() | 'closed'; +%% (Socket, GetRequest, Name) -> {'ok', DevMap} | {'error', Reason} when +%% Socket :: socket(), +%% GetRequest :: 'gifmap', +%% Name :: string(), +%% DevMap :: ioctl_device_map(), +%% Reason :: posix() | 'closed'. + +-spec ioctl(Socket, GetRequest, NameOrIndex) -> {'ok', Result} | {'error', Reason} when + Socket :: socket(), + GetRequest :: 'gifname' | 'gifindex' | + 'gifaddr' | 'gifdstaddr' | 'gifbrdaddr' | + 'gifnetmask' | 'gifhwaddr' | + 'gifmtu' | 'giftxqlen' | 'gifflags', + NameOrIndex :: string() | integer(), + Result :: term(), + Reason :: posix() | 'closed'. + +ioctl(?socket(SockRef), gifname = GetRequest, Index) + when is_integer(Index) -> + prim_socket:ioctl(SockRef, GetRequest, Index); +ioctl(?socket(SockRef), gifindex = GetRequest, Name) + when is_list(Name) -> + prim_socket:ioctl(SockRef, GetRequest, Name); +ioctl(?socket(SockRef), gifaddr = GetRequest, Name) + when is_list(Name) -> + prim_socket:ioctl(SockRef, GetRequest, Name); +ioctl(?socket(SockRef), gifdstaddr = GetRequest, Name) + when is_list(Name) -> + prim_socket:ioctl(SockRef, GetRequest, Name); +ioctl(?socket(SockRef), gifbrdaddr = GetRequest, Name) + when is_list(Name) -> + prim_socket:ioctl(SockRef, GetRequest, Name); +ioctl(?socket(SockRef), gifnetmask = GetRequest, Name) + when is_list(Name) -> + prim_socket:ioctl(SockRef, GetRequest, Name); +ioctl(?socket(SockRef), gifmtu = GetRequest, Name) + when is_list(Name) -> + prim_socket:ioctl(SockRef, GetRequest, Name); +ioctl(?socket(SockRef), gifhwaddr = GetRequest, Name) + when is_list(Name) -> + prim_socket:ioctl(SockRef, GetRequest, Name); +ioctl(?socket(SockRef), giftxqlen = GetRequest, Name) + when is_list(Name) -> + prim_socket:ioctl(SockRef, GetRequest, Name); +ioctl(?socket(SockRef), gifflags = GetRequest, Name) + when is_list(Name) -> + prim_socket:ioctl(SockRef, GetRequest, Name); +ioctl(?socket(SockRef), gifmap = GetRequest, Name) + when is_list(Name) -> + prim_socket:ioctl(SockRef, GetRequest, Name); +ioctl(Socket, GetRequest, Arg) -> + erlang:error(badarg, [Socket, GetRequest, Arg]). + + +-spec ioctl(Socket, SetRequest, Name, Value) -> 'ok' | {'error', Reason} when + Socket :: socket(), + SetRequest :: 'sifflags' | + 'sifaddr' | 'sifdstaddr' | 'sifbrdaddr' | + 'sifnetmask' | 'sifhwaddr' | + 'gifmtu' | 'siftxqlen', + Name :: string(), + Value :: term(), + Reason :: posix() | 'closed'. + +ioctl(?socket(SockRef), sifflags = SetRequest, Name, Flags) + when is_list(Name) andalso is_map(Flags) -> + prim_socket:ioctl(SockRef, SetRequest, Name, Flags); +ioctl(?socket(SockRef), sifaddr = SetRequest, Name, Addr) + when is_list(Name) andalso is_map(Addr) -> + prim_socket:ioctl(SockRef, SetRequest, Name, prim_socket:enc_sockaddr(Addr)); +ioctl(?socket(SockRef), sifdstaddr = SetRequest, Name, DstAddr) + when is_list(Name) andalso is_map(DstAddr) -> + prim_socket:ioctl(SockRef, SetRequest, Name, prim_socket:enc_sockaddr(DstAddr)); +ioctl(?socket(SockRef), sifbrdaddr = SetRequest, Name, BrdAddr) + when is_list(Name) andalso is_map(BrdAddr) -> + prim_socket:ioctl(SockRef, SetRequest, Name, prim_socket:enc_sockaddr(BrdAddr)); +ioctl(?socket(SockRef), sifnetmask = SetRequest, Name, NetMask) + when is_list(Name) andalso is_map(NetMask) -> + prim_socket:ioctl(SockRef, SetRequest, Name, prim_socket:enc_sockaddr(NetMask)); +ioctl(?socket(SockRef), sifmtu = SetRequest, Name, MTU) + when is_list(Name) andalso is_integer(MTU) -> + prim_socket:ioctl(SockRef, SetRequest, Name, MTU); +ioctl(?socket(SockRef), siftxqlen = SetRequest, Name, QLen) + when is_list(Name) andalso is_integer(QLen) -> + prim_socket:ioctl(SockRef, SetRequest, Name, QLen); +ioctl(Socket, SetRequest, Arg1, Arg2) -> + erlang:error(badarg, [Socket, SetRequest, Arg1, Arg2]). + + %% =========================================================================== %% %% cancel - cancel an operation resulting in a select diff --git a/lib/kernel/test/gen_tcp_api_SUITE.erl b/lib/kernel/test/gen_tcp_api_SUITE.erl index debcf2ec71..3d987a180f 100644 --- a/lib/kernel/test/gen_tcp_api_SUITE.erl +++ b/lib/kernel/test/gen_tcp_api_SUITE.erl @@ -571,37 +571,72 @@ do_t_fdconnect(Config) -> ?SKIPT("failed loading util nif lib") end, ?P("try create listen socket"), - L = case gen_tcp:listen(0, - ?INET_BACKEND_OPTS(Config) ++ [{active, false}]) of + LOpts = ?INET_BACKEND_OPTS(Config) ++ [{active, false}], + L = try gen_tcp:listen(0, LOpts) of {ok, LSock} -> LSock; {error, eaddrnotavail = LReason} -> - ?SKIPT(listen_failed_str(LReason)) + ?SKIPT(listen_failed_str(LReason)); + {error, LReason} -> + ?P("UNEXPECTED ERROR - listen error: " + "~n COpts: ~p" + "~n Reason: ~p", [LReason]), + ct:fail({unexpected_listen_error, LReason, LOpts}) + catch + LC : LE : LS -> + ?P("UNEXPECTED ERROR - catched listen: " + "~n LOpts: ~p" + "~n C: ~p" + "~n E: ~p" + "~n S: ~p", [LOpts, LC, LE, LS]), + ct:fail({listen_failure, {LC, LE, LS}, LOpts}) end, - {ok, Port} = inet:port(L), + {ok, LPort} = inet:port(L), ?P("try create file descriptor"), FD = gen_tcp_api_SUITE:getsockfd(), - ?P("try connect using file descriptor ~w", [FD]), - Client = case gen_tcp:connect(localhost, Port, - ?INET_BACKEND_OPTS(Config) ++ - [{fd, FD}, - {active, false}]) of + ?P("try connect (to port ~w) using file descriptor ~w", [LPort, FD]), + COpts = ?INET_BACKEND_OPTS(Config) ++ [{fd, FD}, {active, false}], + Client = try gen_tcp:connect(localhost, LPort, COpts) of {ok, CSock} -> CSock; {error, eaddrnotavail = CReason} -> gen_tcp:close(L), gen_tcp_api_SUITE:closesockfd(FD), - ?SKIPT(connect_failed_str(CReason)) + ?SKIPT(connect_failed_str(CReason)); + {error, CReason} -> + ?P("UNEXPECTED ERROR - connect error: " + "~n COpts: ~p" + "~n Reason: ~p", [COpts, CReason]), + ct:fail({unexpected_connect_error, CReason, COpts}) + catch + CC : CE : CS -> + ?P("UNEXPECTED ERROR - catched connect: " + "~n COpts: ~p" + "~n C: ~p" + "~n E: ~p" + "~n S: ~p", [COpts, CC, CE, CS]), + ct:fail({connect_failure, {CC, CE, CS}, COpts}) end, ?P("try accept connection"), - Server = case gen_tcp:accept(L) of + Server = try gen_tcp:accept(L) of {ok, ASock} -> ASock; {error, eaddrnotavail = AReason} -> gen_tcp:close(Client), gen_tcp:close(L), gen_tcp_api_SUITE:closesockfd(FD), - ?SKIPT(accept_failed_str(AReason)) + ?SKIPT(accept_failed_str(AReason)); + {error, AReason} -> + ?P("UNEXPECTED ERROR - accept error: " + "~n Reason: ~p", [AReason]), + ct:fail({unexpected_accept_error, AReason}) + catch + AC : AE : AS -> + ?P("UNEXPECTED ERROR - catched accept: " + "~n C: ~p" + "~n E: ~p" + "~n S: ~p", [AC, AE, AS]), + ct:fail({accept_failure, {AC, AE, AS}}) end, ?P("begin validation"), ok = gen_tcp:send(Client, Question), diff --git a/lib/kernel/test/gen_tcp_misc_SUITE.erl b/lib/kernel/test/gen_tcp_misc_SUITE.erl index 9f23ca5992..f8561a44c4 100644 --- a/lib/kernel/test/gen_tcp_misc_SUITE.erl +++ b/lib/kernel/test/gen_tcp_misc_SUITE.erl @@ -6563,13 +6563,17 @@ delay_send_error(Config) -> ?LISTEN(Config, 0, [{reuseaddr, true}, {packet, 1}, {active, false}]), {ok,{{0,0,0,0},PortNum}}=inet:sockname(L), - ?P("try connect - with delay_send:true"), + + delay_send_error(Config, L, PortNum, false), + delay_send_error(Config, L, PortNum, true). +delay_send_error(Config, L, PortNum, Active) -> + ?P("try connect - with delay_send:true active:~p",[Active]), {ok, C} = ?CONNECT(Config, "localhost", PortNum, - [{packet, 1}, {active, false}, {delay_send, true}]), + [{packet, 1}, {active, Active}, {delay_send, true}]), ?P("try accept"), - {ok, S} = gen_tcp:accept(L), + {ok, A} = gen_tcp:accept(L), %% Do a couple of sends first to see that it works ?P("send data"), ok = gen_tcp:send(C, "hello"), @@ -6579,7 +6583,7 @@ delay_send_error(Config) -> ok = gen_tcp:send(C, "hello"), %% Close the receiver ?P("close receiver (accepted socket)"), - ok = gen_tcp:close(S), + ok = gen_tcp:shutdown(C, write), %% ?P("send data"), case gen_tcp:send(C, "hello") of @@ -6596,6 +6600,9 @@ delay_send_error(Config) -> ?P("closed (expected)"), ok end, + ?P("close accepted socket"), + ok = gen_tcp:close(A), + ?P("close connected socket"), ok = gen_tcp:close(C). diff --git a/lib/kernel/test/gen_udp_SUITE.erl b/lib/kernel/test/gen_udp_SUITE.erl index cc726e8ace..0317f0fe8d 100644 --- a/lib/kernel/test/gen_udp_SUITE.erl +++ b/lib/kernel/test/gen_udp_SUITE.erl @@ -1702,17 +1702,18 @@ do_reconnect(Config) -> {ok, {LocalAddr,Port}} = inet:sockname(S), ok = inet:close(S). -%% For Linux to behave predictably we need to bind -%% to a specific port; when we bind to port 0 -%% and get an ephemeral port - it apparently can change -%% when we reconnect to a different destination. +%% For Linux to keep the port when we reconnect; +%% we need to first bind to a specific port. +%% If we bind to port 0 and get an ephemeral port +%% it apparently can change when we reconnect to a different +%% destination depending on routing and interfaces. %% %% I consider this a workaround for a Linux bug, %% ironically in a test case that tests %% a workaround for another Linux bug (related)... %% open_port_0(Config, Opts) -> -open_port_0(Config, 0, Opts, 10). + open_port_0(Config, 0, Opts, 10). %% open_port_0(Config, Port, Opts, N) -> case ?OPEN(Config, Port, Opts) of @@ -1742,22 +1743,20 @@ do_implicit_inet6(Config) -> Host = ok(inet:gethostname()), case inet:getaddr(Host, inet6) of {ok, {16#fe80,0,0,0,_,_,_,_} = Addr} -> - {skip, - "Got link local IPv6 address: " - ++inet:ntoa(Addr)}; + ?SKIPT("Got link local IPv6 address: " + ++inet:ntoa(Addr)); {ok, Addr} -> implicit_inet6(Config, Host, Addr); {error, Reason} -> - {skip, - "Can not look up IPv6 address: " - ++atom_to_list(Reason)} + ?SKIPT("Can not look up IPv6 address: " + ++atom_to_list(Reason)) end. implicit_inet6(Config, Host, Addr) -> Active = {active,false}, Loopback = {0,0,0,0,0,0,0,1}, ?P("try 1 with explit inet6 on loopback"), - S1 = case open_port_0(Config, [inet6, Active, {ip, Loopback}]) of + S1 = case ?OPEN(Config, 0, [inet6, Active, {ip, Loopback}]) of {ok, Sock1} -> Sock1; {error, eaddrnotavail = Reason1} -> @@ -1770,7 +1769,7 @@ implicit_inet6(Config, Host, Addr) -> %% Localaddr = ok(get_localaddr()), ?P("try 2 on local addr (~p)", [Localaddr]), - S2 = case open_port_0(Config, [{ip, Localaddr}, Active]) of + S2 = case ?OPEN(Config, 0, [{ip, Localaddr}, Active]) of {ok, Sock2} -> Sock2; {error, eaddrnotavail = Reason2} -> @@ -1780,7 +1779,7 @@ implicit_inet6(Config, Host, Addr) -> ok = gen_udp:close(S2), %% ?P("try 3 on addr ~p (~p)", [Addr, Host]), - S3 = case open_port_0(Config, [{ifaddr, Addr}, Active]) of + S3 = case ?OPEN(Config, 0, [{ifaddr, Addr}, Active]) of {ok, Sock3} -> Sock3; {error, eaddrnotavail = Reason3} -> diff --git a/lib/kernel/test/inet_SUITE.erl b/lib/kernel/test/inet_SUITE.erl index 838e0971fe..cf3ff47325 100644 --- a/lib/kernel/test/inet_SUITE.erl +++ b/lib/kernel/test/inet_SUITE.erl @@ -50,7 +50,8 @@ parse_strict_address/1, ipv4_mapped_ipv6_address/1, ntoa/1, simple_netns/1, simple_netns_open/1, add_del_host/1, add_del_host_v6/1, - simple_bind_to_device/1, simple_bind_to_device_open/1 + simple_bind_to_device/1, simple_bind_to_device_open/1, + socknames_sctp/1, socknames_tcp/1, socknames_udp/1 ]). -export([ @@ -79,12 +80,14 @@ all() -> getifaddrs, parse_strict_address, ipv4_mapped_ipv6_address, ntoa, simple_netns, simple_netns_open, add_del_host, add_del_host_v6, - simple_bind_to_device, simple_bind_to_device_open + simple_bind_to_device, simple_bind_to_device_open, + {group, socknames} ]. groups() -> [ - {parse, [], parse_cases()} + {parse, [], parse_cases()}, + {socknames, [], socknames_cases()} ]. parse_cases() -> @@ -93,6 +96,13 @@ parse_cases() -> parse_address ]. +socknames_cases() -> + [ + socknames_sctp, + socknames_tcp, + socknames_udp + ]. + %% Required configuaration required(v4) -> [{require, test_host_ipv4_only}, @@ -1879,6 +1889,140 @@ add_del_host_v6(_Config) -> {ok, HostEnt} = inet_hosts:gethostbyname(Name, inet6). + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +%%% This is a supremely simple test case. +%%% It basically just tests that the function (inet:socknames/1,2) +%%% returns with the expected result (a list of addresses, which in +%%% the case of tcp and udp should be exactly one address long). + +socknames_sctp(Config) when is_list(Config) -> + %%% ?TC_TRY(?FUNCTION_NAME, fun() -> do_socknames_sctp(Config) end). + {skip, not_implemented_yet}. + + +socknames_tcp(Config) when is_list(Config) -> + ?TC_TRY(?FUNCTION_NAME, fun() -> do_socknames_tcp0(Config) end). + +do_socknames_tcp0(_Config) -> + %% Begin with a the plain old boring (= port) socket(s) + ?P("Test socknames for 'old' socket (=port)"), + do_socknames_tcp1([]), + + %% And *maybe* also check the 'new' shiny socket sockets + try socket:info() of + #{} -> + ?P("Test socknames for 'new' socket (=socket nif)"), + do_socknames_tcp1([{inet_backend, socket}]) + catch + error : notsup -> + ?P("Skip test of socknames for 'new' socket (=socket nif)"), + ok + end. + + +do_socknames_tcp1(Conf) -> + ?P("try create listen socket"), + {ok, S1} = gen_tcp:listen(0, Conf), + ?P("try get socknames for (listen) socket: " + "~n ~p", [S1]), + PortNumber1 = case inet:socknames(S1) of + {ok, [{_Addr1, PortNo1}]} -> + PortNo1; + {ok, Addrs1} -> + ?P("failed socknames: unexpected number of addresses" + "~n ~p", [Addrs1]), + exit({unexpected_addrs_length, length(Addrs1)}); + {error, Reason1} -> + ?P("failed socknames: error" + "~n ~p", [Reason1]), + exit({skip, {listen_socket, Reason1}}) + end, + ?P("try connect to listen socket on port ~p", [PortNumber1]), + {ok, S2} = gen_tcp:connect("localhost", PortNumber1, Conf), + ?P("try get socknames for (connected) socket: " + "~n ~p", [S2]), + case inet:socknames(S2) of + {ok, [_Addr2]} -> + ok; + {ok, Addrs2} -> + ?P("failed socknames: unexpected number of addresses" + "~n ~p", [Addrs2]), + exit({unexpected_addrs_length, length(Addrs2)}); + {error, Reason2} -> + ?P("failed socknames: error" + "~n ~p", [Reason2]), + exit({skip, {connected_socket, Reason2}}) + end, + ?P("try accept connection"), + {ok, S3} = gen_tcp:accept(S1), + ?P("try get socknames for (accepted) socket: " + "~n ~p", [S3]), + case inet:socknames(S3) of + {ok, [_Addr3]} -> + ok; + {ok, Addrs3} -> + ?P("failed socknames: unexpected number of addresses" + "~n ~p", [Addrs3]), + exit({unexpected_addrs_length, length(Addrs3)}); + {error, Reason3} -> + ?P("failed socknames: error" + "~n ~p", [Reason3]), + exit({skip, {accepted_socket, Reason3}}) + end, + ?P("close socket(s)"), + (catch gen_tcp:close(S3)), + (catch gen_tcp:close(S2)), + (catch gen_tcp:close(S1)), + ?P("done"), + ok. + + +socknames_udp(Config) when is_list(Config) -> + ?TC_TRY(?FUNCTION_NAME, fun() -> do_socknames_udp0(Config) end). + +do_socknames_udp0(_Config) -> + %% Begin with a the plain old boring (= port) socket(s) + ?P("Test socknames for 'old' socket (=port)"), + do_socknames_udp1([]), + + %% And *maybe* also check the 'new' shiny socket sockets + try socket:info() of + #{} -> + ?P("Test socknames for 'new' socket (=socket nif)"), + do_socknames_udp1([{inet_backend, socket}]) + catch + error : notsup -> + ?P("Skip test of socknames for 'new' socket (=socket nif)"), + ok + end. + + +do_socknames_udp1(Conf) -> + ?P("try create socket"), + {ok, S1} = gen_udp:open(0, Conf), + ?P("try get socknames for socket: " + "~n ~p", [S1]), + case inet:socknames(S1) of + {ok, [_Addr1]} -> + ok; + {ok, Addrs1} -> + ?P("failed socknames: unexpected number of addresses" + "~n ~p", [Addrs1]), + exit({unexpected_addrs_length, length(Addrs1)}); + {error, Reason1} -> + ?P("failed socknames: error" + "~n ~p", [Reason1]), + exit({skip, {listen_socket, Reason1}}) + end, + ?P("close socket"), + (catch gen_udp:close(S1)), + ?P("done"), + ok. + + + %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% pi(Item) -> diff --git a/lib/kernel/test/logger_SUITE.erl b/lib/kernel/test/logger_SUITE.erl index 5545759b5d..c3a4a0417a 100644 --- a/lib/kernel/test/logger_SUITE.erl +++ b/lib/kernel/test/logger_SUITE.erl @@ -1004,7 +1004,7 @@ app_config(Config) -> ok. -%% This test case is maintly to see code coverage. Note that +%% This test case is mainly to see code coverage. Note that %% logger_env_var_SUITE tests a lot of the same, and checks the %% functionality more thoroughly, but since it all happens at node %% start, it is not possible to see code coverage in that test. diff --git a/lib/kernel/test/logger_env_var_SUITE.erl b/lib/kernel/test/logger_env_var_SUITE.erl index 494059e9e3..e67d8551cd 100644 --- a/lib/kernel/test/logger_env_var_SUITE.erl +++ b/lib/kernel/test/logger_env_var_SUITE.erl @@ -60,7 +60,8 @@ groups() -> logger_many_handlers_default_first, logger_many_handlers_default_last, logger_many_handlers_default_last_broken_filter, - logger_proxy + logger_proxy, + logger_metadata ]}, {bad,[],[bad_error_logger, bad_level, @@ -81,6 +82,7 @@ all() -> default(Config) -> {ok,#{primary:=P,handlers:=Hs,module_levels:=ML},_Node} = setup(Config,[]), notice = maps:get(level,P), + true = #{} == maps:get(metadata,P), #{module:=logger_std_h} = StdC = find(?STANDARD_HANDLER,Hs), all = maps:get(level,StdC), StdFilters = maps:get(filters,StdC), @@ -555,6 +557,12 @@ logger_proxy(Config) -> ok. +logger_metadata(Config) -> + {ok,#{ primary := #{ metadata := #{ test := test } } }, _} + = setup(Config, [{logger_metadata,#{ test => test }}]), + + ok. + sasl_compatible_false(Config) -> Log = file(Config,?FUNCTION_NAME), {ok,_,Node} = setup(Config, diff --git a/lib/kernel/test/socket_SUITE.erl b/lib/kernel/test/socket_SUITE.erl index 5cecd308f3..5854952049 100644 --- a/lib/kernel/test/socket_SUITE.erl +++ b/lib/kernel/test/socket_SUITE.erl @@ -258,6 +258,7 @@ monitor_open_and_exit_multi_mon/1, monitor_open_and_close_multi_socks_and_mon/1, monitor_open_and_exit_multi_socks_and_mon/1, + monitor_closed_socket/1, %% *** Socket Closure *** sc_cpe_socket_cleanup_tcp4/1, @@ -297,6 +298,22 @@ sc_rs_recvmsg_send_shutdown_receive_tcp6/1, sc_rs_recvmsg_send_shutdown_receive_tcpL/1, + %% Socket IOCTL simple + ioctl_simple1/1, + %% Socket IOCTL get requests + ioctl_get_gifname/1, + ioctl_get_gifindex/1, + ioctl_get_gifaddr/1, + ioctl_get_gifdstaddr/1, + ioctl_get_gifbrdaddr/1, + ioctl_get_gifnetmask/1, + ioctl_get_gifmtu/1, + ioctl_get_gifhwaddr/1, + ioctl_get_giftxqlen/1, + ioctl_get_gifflags/1, + ioctl_get_gifmap/1, + %% ioctl_set_requests/1, + %% *** Traffic *** traffic_send_and_recv_counters_tcp4/1, traffic_send_and_recv_counters_tcp6/1, @@ -710,6 +727,7 @@ all() -> Groups = [{api, "ESOCK_TEST_API", include}, {reg, undefined, include}, {monitor, undefined, include}, + {ioctl, undefined, include}, {socket_close, "ESOCK_TEST_SOCK_CLOSE", include}, {traffic, "ESOCK_TEST_TRAFFIC", include}, {ttest, "ESOCK_TEST_TTEST", exclude}, @@ -769,6 +787,10 @@ groups() -> {sc_local_close, [], sc_lc_cases()}, {sc_remote_close, [], sc_rc_cases()}, {sc_remote_shutdown, [], sc_rs_cases()}, + {ioctl, [], ioctl_cases()}, + {ioctl_simple, [], ioctl_simple_cases()}, + {ioctl_get, [], ioctl_get_cases()}, + {ioctl_set, [], ioctl_set_cases()}, {traffic, [], traffic_cases()}, {traffic_counters, [], traffic_counters_cases()}, {traffic_chunks, [], traffic_chunks_cases()}, @@ -1132,7 +1154,8 @@ monitor_cases() -> monitor_open_and_close_multi_mon, monitor_open_and_exit_multi_mon, monitor_open_and_close_multi_socks_and_mon, - monitor_open_and_exit_multi_socks_and_mon + monitor_open_and_exit_multi_socks_and_mon, + monitor_closed_socket ]. @@ -1207,6 +1230,41 @@ sc_rs_cases() -> ]. +ioctl_cases() -> + [ + {group, ioctl_simple}, + {group, ioctl_get}, + {group, ioctl_set} + ]. + + +ioctl_simple_cases() -> + [ + ioctl_simple1 + ]. + + +ioctl_get_cases() -> + [ + ioctl_get_gifname, + ioctl_get_gifindex, + ioctl_get_gifaddr, + ioctl_get_gifdstaddr, + ioctl_get_gifbrdaddr, + ioctl_get_gifnetmask, + ioctl_get_gifmtu, + ioctl_get_gifhwaddr, + ioctl_get_giftxqlen, + ioctl_get_gifflags, + ioctl_get_gifmap + ]. + + +ioctl_set_cases() -> + [ + ]. + + traffic_cases() -> [ {group, traffic_counters}, @@ -25861,6 +25919,46 @@ reg_s_single_open_and_close_and_count() -> reg_si_fail(wrong_number_of_sockets2, {N, NumberOf2}) end, + + i("verify owner sockets (none)", []), + Expected1 = [], + case socket:which_sockets(self()) of + Expected1 -> + ok; + Unexpected1 -> + reg_si_fail(wrong_sockets_own1, {Expected1, Unexpected1}) + end, + + i("create some sockets", []), + OwnSockets = lists:sort( + [fun({D, T})-> + i("create ~w:~w socket", [D, T]), + {ok, OS} = socket:open(D, T, default), + OS + end(SockInfo) || SockInfo <- + [{inet, dgram}, + {inet, dgram}, + {inet, stream}, + {inet, stream}]]), + + i("verify owner sockets (~w)", [length(OwnSockets)]), + case lists:sort(socket:which_sockets(self())) of + OwnSockets -> + ok; + Unexpected2 -> + reg_si_fail(wrong_sockets_own2, {OwnSockets, Unexpected2}) + end, + + i("close (own) sockets"), + lists:foreach(fun(S) -> + i("close socket"), + ok = socket:close(S) + end, OwnSockets), + ?SLEEP(1000), + + i("verify number of sockets(2) (again)"), + NumberOf2 = socket:number_of(), + i("verify pre-existing sockets(2)", []), case socket:which_sockets() of Existing -> @@ -30713,6 +30811,299 @@ mon_open_and_exit_multi_socks_and_mon(InitState) -> ok = ?SEV_AWAIT_FINISH([Owner, Tester]). + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% +%% Create several sockets (by 'owner' process), monitor each from several +%% different processes, then exit the owner. +%% The processes that did the monitor shall receive one socket DOWN for +%% each socket. + +monitor_closed_socket(suite) -> + []; +monitor_closed_socket(doc) -> + []; +monitor_closed_socket(_Config) when is_list(_Config) -> + ?TT(?SECS(5)), + tc_try(monitor_closed_socket, + fun() -> + InitState = #{domain => inet, + type => stream, + protocol => tcp}, + ok = mon_closed_socket(InitState) + end). + + +mon_closed_socket(InitState) -> + OwnerSeq = + [ + %% *** Wait for start order part *** + #{desc => "await start (from tester)", + cmd => fun(State) -> + Tester = ?SEV_AWAIT_START(), + {ok, State#{tester => Tester}} + end}, + #{desc => "monitor tester", + cmd => fun(#{tester := Tester} = _State) -> + _MRef = erlang:monitor(process, Tester), + ok + end}, + + %% *** Init part *** + #{desc => "create socket", + cmd => fun(#{domain := Domain, + type := Type, + protocol := Proto} = State) -> + case socket:open(Domain, Type, Proto) of + {ok, Sock} -> + {ok, State#{sock => Sock}}; + {error, _} = ERROR -> + ERROR + end + end}, + #{desc => "announce ready (init)", + cmd => fun(#{tester := Tester, + sock := Sock} = State) -> + ?SEV_ANNOUNCE_READY(Tester, init, Sock), + (catch socket:close(Sock)), + {ok, maps:remove(sock, State)} + end}, + + %% The actual test + #{desc => "await terminate (from tester)", + cmd => fun(#{tester := Tester} = State) -> + case ?SEV_AWAIT_TERMINATE(Tester, tester) of + ok -> + {ok, maps:remove(tester, State)}; + {error, _} = ERROR -> + ERROR + end + end}, + + %% *** We are done *** + ?SEV_FINISH_NORMAL + ], + + + ClientSeq = + [ + %% *** Wait for start order part *** + #{desc => "await start (from tester)", + cmd => fun(State) -> + Tester = ?SEV_AWAIT_START(), + {ok, State#{tester => Tester}} + end}, + #{desc => "monitor tester", + cmd => fun(#{tester := Tester} = _State) -> + _MRef = erlang:monitor(process, Tester), + ok + end}, + + %% *** Init part *** + #{desc => "announce ready (init)", + cmd => fun(#{tester := Tester} = _State) -> + ?SEV_ANNOUNCE_READY(Tester, init), + ok + end}, + #{desc => "await continue (socket)", + cmd => fun(#{tester := Tester} = State) -> + {ok, Sock} = + ?SEV_AWAIT_CONTINUE(Tester, tester, socket), + ?SEV_IPRINT("Socket: " + "~n ~p", [Sock]), + {ok, State#{sock => Sock}} + end}, + + + %% The actual test + #{desc => "await continue (monitor)", + cmd => fun(#{tester := Tester}) -> + ?SEV_AWAIT_CONTINUE(Tester, tester, monitor) + end}, + + #{desc => "monitor socket", + cmd => fun(#{sock := Sock} = State) -> + MRef = socket:monitor(Sock), + ?SEV_IPRINT("Monitors: " + "~n ~p", [MRef]), + {ok, State#{mon => MRef}} + end}, + + #{desc => "announce ready (monitor)", + cmd => fun(#{tester := Tester}) -> + ?SEV_ANNOUNCE_READY(Tester, monitor), + ok + end}, + + #{desc => "await continue (down)", + cmd => fun(#{tester := Tester}) -> + ?SEV_AWAIT_CONTINUE(Tester, tester, down) + end}, + + #{desc => "await socket down", + cmd => fun(#{sock := Sock, + mon := MRef} = State) -> + receive + {'DOWN', MRef, socket, Sock, Info} -> + ?SEV_IPRINT("received expected down: " + "~n MRef: ~p" + "~n Socket: ~p" + "~n Info: ~p", + [MRef, Sock, Info]), + State2 = maps:remove(mon, State), + State3 = maps:remove(sock, State2), + {ok, State3} + after 1000 -> + ?SEV_EPRINT("socket down timeout"), + {error, timeout} + end + end}, + + #{desc => "announce ready (down)", + cmd => fun(#{tester := Tester}) -> + ?SEV_ANNOUNCE_READY(Tester, down), + ok + end}, + + + #{desc => "await terminate (from tester)", + cmd => fun(#{tester := Tester} = State) -> + case ?SEV_AWAIT_TERMINATE(Tester, tester) of + ok -> + {ok, maps:remove(tester, State)}; + {error, _} = ERROR -> + ERROR + end + end}, + + %% *** We are done *** + ?SEV_FINISH_NORMAL + ], + + TesterSeq = + [ + %% *** Init part *** + #{desc => "monitor 'owner'", + cmd => fun(#{owner := Owner} = _State) -> + _MRef = erlang:monitor(process, Owner), + ok + end}, + #{desc => "order (owner) start", + cmd => fun(#{owner := Pid} = _State) -> + ?SEV_ANNOUNCE_START(Pid), + ok + end}, + #{desc => "await (owner) ready", + cmd => fun(#{owner := Pid} = State) -> + {ok, Sock} = ?SEV_AWAIT_READY(Pid, owner, init), + {ok, State#{sock => Sock}} + end}, + + #{desc => "monitor client", + cmd => fun(#{client := Pid} = _State) -> + _MRef = erlang:monitor(process, Pid), + ok + end}, + #{desc => "order client start", + cmd => fun(#{client := Pid} = _State) -> + ?SEV_ANNOUNCE_START(Pid), + ok + end}, + #{desc => "await client ready", + cmd => fun(#{client := Pid} = _State) -> + ok = ?SEV_AWAIT_READY(Pid, client, init) + end}, + #{desc => "send socket to client", + cmd => fun(#{client := Pid, + sock := Sock} = _State) -> + ?SEV_ANNOUNCE_CONTINUE(Pid, socket, Sock), + ok + end}, + + %% The actual test + %% There is no way we can actually know that the "system" + %% does not create *any* sockets... + #{desc => "order client to monitor", + cmd => fun(#{client := Pid} = _State) -> + ?SEV_ANNOUNCE_CONTINUE(Pid, monitor), + ok + end}, + #{desc => "await client ready", + cmd => fun(#{client := Pid} = _State) -> + ok = ?SEV_AWAIT_READY(Pid, client, monitor) + end}, + + #{desc => "order client to await down", + cmd => fun(#{client := Pid} = _State) -> + ?SEV_ANNOUNCE_CONTINUE(Pid, down), + ok + end}, + + ?SEV_SLEEP(?SECS(1)), + + #{desc => "order (owner) terminate", + cmd => fun(#{owner := Pid} = _State) -> + ?SEV_ANNOUNCE_TERMINATE(Pid), + ok + end}, + #{desc => "await (owner) termination", + cmd => fun(#{owner := Pid} = State) -> + case ?SEV_AWAIT_TERMINATION(Pid) of + ok -> + {ok, maps:remove(owner, State)}; + {error, _} = ERROR -> + ERROR + end + end}, + + #{desc => "await client ready", + cmd => fun(#{client := Pid} = _State) -> + ok = ?SEV_AWAIT_READY(Pid, client, down) + end}, + + #{desc => "verify total number of monitors (=0)", + cmd => fun(_State) -> + 0 = socket:number_of_monitors(), + ok + end}, + + ?SEV_SLEEP(?SECS(1)), + + %% Cleanup + #{desc => "order client terminate", + cmd => fun(#{client := Pid} = _State) -> + ?SEV_ANNOUNCE_TERMINATE(Pid), + ok + end}, + #{desc => "await client termination", + cmd => fun(#{client := Pid} = State) -> + case ?SEV_AWAIT_TERMINATION(Pid) of + ok -> + {ok, maps:remove(client, State)}; + {error, _} = ERROR -> + ERROR + end + end}, + + %% *** We are done *** + ?SEV_FINISH_NORMAL + ], + + i("start (socket) owner evaluator"), + Owner = ?SEV_START("owner", OwnerSeq, InitState), + + i("start client evaluator"), + Client = ?SEV_START("client", ClientSeq, InitState), + + i("start tester evaluator"), + TesterInitState = #{owner => Owner#ev.pid, + client => Client#ev.pid}, + Tester = ?SEV_START("tester", TesterSeq, TesterInitState), + + i("await evaluator"), + ok = ?SEV_AWAIT_FINISH([Owner, Client, Tester]). + + + %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %% %% @@ -34792,6 +35183,886 @@ sc_rs_recvmsg_send_shutdown_receive_tcpL(_Config) when is_list(_Config) -> end). + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% +%% This test case is intended to (simply) test "some" ioctl features. +%% + +ioctl_simple1(suite) -> + []; +ioctl_simple1(doc) -> + []; +ioctl_simple1(_Config) when is_list(_Config) -> + ?TT(?SECS(5)), + tc_try(?FUNCTION_NAME, + fun() -> + has_support_ioctl_requests(), + has_support_ioctl_gifconf() + end, + fun() -> + InitState = #{}, + ok = do_ioctl_simple(InitState) + end). + + +do_ioctl_simple(_State) -> + i("create dummy dgram:UDP socket"), + {ok, Sock1} = socket:open(inet, dgram, udp), + i("perform simple ioctl (expect success)"), + case socket:ioctl(Sock1, gifconf) of + {ok, IfConf1} when is_list(IfConf1) -> + i("=> success"), + ok; + {error, Reason1} -> + i("error: unexpected failure: " + "~n ~p", [Reason1]), + exit({unexpected_ioctl_failure, dgram, Reason1}); + Else11 -> + i("error: unexpected result: " + "~n ~p", [Else11]), + exit({unexpected_ioctl_result, dgram, Else11}) + end, + i("close dummy dgram:UDP socket"), + ok = socket:close(Sock1), + i("perform simple ioctl (expect failure)"), + case socket:ioctl(Sock1, gifconf) of + {error, closed} -> + i("=> success"), + ok; + Else12 -> + i("error: unexpected result: " + "~n ~p", [Else12]), + exit({unexpected_ioctl_result, dgram, Else12}) + end, + + i("create dummy stream:TCP socket"), + {ok, Sock2} = socket:open(inet, stream, tcp), + i("perform simple ioctl (expect success)"), + case socket:ioctl(Sock2, gifconf) of + {ok, IfConf2} when is_list(IfConf2) -> + i("=> success"), + ok; + {error, Reason2} -> + i("error: unexpected result: " + "~n ~p", [Reason2]), + exit({unexpected_ioctl_failure, stream, Reason2}); + Else21 -> + i("error: unexpected result: " + "~n ~p", [Else21]), + exit({unexpected_ioctl_result, stream, Else21}) + end, + i("close dummy stream:TCP socket"), + ok = socket:close(Sock2), + i("perform simple ioctl (expect failure)"), + case socket:ioctl(Sock2, gifconf) of + {error, closed} -> + i("=> success"), + ok; + Else22 -> + i("error: unexpected result: " + "~n ~p", [Else22]), + exit({unexpected_ioctl_result, stream, Else22}) + end, + + i("done"), + ok. + + + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% +%% These test case(s) are intended to (simply) test "some" ioctl get +%% request(s). +%% + +ioctl_get_gifname(suite) -> + []; +ioctl_get_gifname(doc) -> + []; +ioctl_get_gifname(_Config) when is_list(_Config) -> + ?TT(?SECS(5)), + tc_try(?FUNCTION_NAME, + fun() -> + has_support_net_if_names(), + has_support_ioctl_gifname() + end, + fun() -> + InitState = #{}, + ok = do_ioctl_get_gifname(InitState) + end). + + +do_ioctl_get_gifname(_State) -> + i("create dummy dgram:UDP socket"), + {ok, Sock} = socket:open(inet, dgram, udp), + + i("get if names"), + {ok, IfNames} = net:if_names(), + + i("try ioctl all if names: " + "~n ~p", [IfNames]), + _ = [case socket:ioctl(Sock, gifname, IfIdx) of + {ok, IfName} -> + i("got expected interface name ~p for index ~w", [IfName, IfIdx]), + ok; + {ok, OtherName} -> + %% Oups?! + i("<ERROR> got unexpected interface name for index ~w" + "~n Expected: ~p" + "~n Received: ~p", [IfIdx, IfName, OtherName]), + socket:close(Sock), + ?FAIL({unexpected_interface_name, IfIdx, OtherName, IfName}); + {error, Reason} -> + i("<ERROR> got unexpected error for interface index ~w" + "~n Reason: ~p", [IfIdx, Reason]), + socket:close(Sock), + ?FAIL({unexpected_failure, IfName, Reason}) + end || {IfIdx, IfName} <- IfNames], + + i("close dummy dgram:UDP socket"), + ok = socket:close(Sock), + + i("done"), + ok. + + +%% --- gifindex --- + +ioctl_get_gifindex(suite) -> + []; +ioctl_get_gifindex(doc) -> + []; +ioctl_get_gifindex(_Config) when is_list(_Config) -> + ?TT(?SECS(5)), + tc_try(?FUNCTION_NAME, + fun() -> + has_support_net_if_names(), + has_support_ioctl_gifindex() + end, + fun() -> + InitState = #{}, + ok = do_ioctl_get_gifindex(InitState) + end). + + +do_ioctl_get_gifindex(_State) -> + i("create dummy dgram:UDP socket"), + {ok, Sock} = socket:open(inet, dgram, udp), + + i("get if names"), + {ok, IfNames} = net:if_names(), + + i("try ioctl all if indexes: " + "~n ~p", [IfNames]), + _ = [case socket:ioctl(Sock, gifindex, IfName) of + {ok, IfIdx} -> + i("got expected interface index ~w for interface ~p", + [IfIdx, IfName]), + ok; + {ok, OtherIdx} -> + %% Oups?! + i("<ERROR> got unexpected interface index for interface ~p" + "~n Expected: ~p" + "~n Received: ~p", [IfName, IfIdx, OtherIdx]), + socket:close(Sock), + ?FAIL({unexpected_interface_index, IfName, OtherIdx, IfIdx}); + {error, Reason} -> + i("<ERROR> got unexpected error for interface ~p" + "~n Reason: ~p", [IfName, Reason]), + socket:close(Sock), + ?FAIL({unexpected_failure, IfIdx, Reason}) + end || {IfIdx, IfName} <- IfNames], + + i("close dummy dgram:UDP socket"), + ok = socket:close(Sock), + + i("done"), + ok. + + + +%% --- gifaddr --- + +ioctl_get_gifaddr(suite) -> + []; +ioctl_get_gifaddr(doc) -> + []; +ioctl_get_gifaddr(_Config) when is_list(_Config) -> + ?TT(?SECS(5)), + tc_try(?FUNCTION_NAME, + fun() -> + has_support_net_if_names(), + has_support_ioctl_gifaddr() + end, + fun() -> + InitState = #{}, + ok = do_ioctl_get_gifaddr(InitState) + end). + + +do_ioctl_get_gifaddr(_State) -> + i("create dummy dgram:UDP socket"), + {ok, Sock} = socket:open(inet, dgram, udp), + + i("get if names"), + {ok, IfNames} = net:if_names(), + + i("try ioctl all if indexes: " + "~n ~p", [IfNames]), + %% This a *very* simple test... + %% ...just to check that we actually get an socket address + _ = [case socket:ioctl(Sock, gifaddr, IfName) of + {ok, #{family := Fam, + addr := Addr}} -> + i("got (expected) socket address for interface ~p (~w): " + "~n (~w) ~p", [IfName, IfIdx, Fam, Addr]), + ok; + {ok, Crap} -> + %% Oups?! + i("<ERROR> got unexpected result for interface ~p (~w)" + "~n ~p", [IfName, IfIdx, Crap]), + socket:close(Sock), + ?FAIL({unexpected_addr, IfName, IfIdx, Crap}); + {error, eaddrnotavail = Reason} -> + i("got unexpected error for interface ~p (~w) => " + "SKIP interface" + "~n Reason: ~p", [IfName, IfIdx, Reason]), + ignore; + {error, eperm = Reason} -> + i("got unexpected error for interface ~p (~w) => " + "SKIP interface" + "~n Reason: ~p", [IfName, IfIdx, Reason]), + ignore; + {error, Reason} -> + i("<ERROR> got unexpected error for interface ~p (~w)" + "~n Reason: ~p", [IfName, IfIdx, Reason]), + socket:close(Sock), + ?FAIL({unexpected_failure, IfName, IfIdx, Reason}) + end || {IfIdx, IfName} <- IfNames], + + i("close dummy dgram:UDP socket"), + ok = socket:close(Sock), + + i("done"), + ok. + + + +%% --- gifdstaddr --- + +ioctl_get_gifdstaddr(suite) -> + []; +ioctl_get_gifdstaddr(doc) -> + []; +ioctl_get_gifdstaddr(_Config) when is_list(_Config) -> + ?TT(?SECS(5)), + tc_try(?FUNCTION_NAME, + fun() -> + has_support_net_if_names(), + has_support_ioctl_gifdstaddr() + end, + fun() -> + InitState = #{}, + ok = do_ioctl_get_gifdstaddr(InitState) + end). + + +do_ioctl_get_gifdstaddr(_State) -> + Domain = inet, + LSA = which_local_socket_addr(Domain), + + i("create and init listen stream:TCP socket"), + {ok, LSock} = socket:open(inet, stream, tcp), + ok = socket:bind(LSock, LSA#{port => 0}), + ok = socket:listen(LSock), + {ok, #{port := LPort}} = socket:sockname(LSock), + + i("create and init connection stream:TCP socket"), + {ok, CSock} = socket:open(inet, stream, tcp), + + i("attempt nowait connect"), + {select, {select_info, _Tag, SelectHandle} = _SelectInfo} = + socket:connect(CSock, LSA#{port => LPort}, nowait), + + i("attempt accept"), + {ok, ASock} = socket:accept(LSock), + + i("await connection ready"), + receive + {'$socket', CSock, select, SelectHandle} -> + ok + end, + + i("attmpt complete connection"), + ok = socket:connect(CSock), + + i("get if names"), + {ok, IfNames} = net:if_names(), + + i("try ioctl all if indexes: " + "~n ~p", [IfNames]), + %% This a *very* simple test... + %% ...just to check that we actually get an socket address + verify_gifdstaddr(ASock, "accept", IfNames), + verify_gifdstaddr(CSock, "connect", IfNames), + verify_gifdstaddr(LSock, "listen", IfNames), + + i("close socket(s)"), + _ = socket:close(CSock), + _ = socket:close(ASock), + _ = socket:close(LSock), + + i("done"), + ok. + + + +verify_gifdstaddr(_Sock, _Prefix, []) -> + ok; +verify_gifdstaddr(Sock, Prefix, [{IfIdx, IfName} | IfNames]) -> + i("[~s] attempt verify gifdstaddr for interface ~s (~w)", + [Prefix, IfName, IfIdx]), + verify_gifdstaddr(Sock, Prefix, IfIdx, IfName), + verify_gifdstaddr(Sock, Prefix, IfNames). + +verify_gifdstaddr(Sock, Prefix, IfIdx, IfName) -> + {OsFam, OsName} = os:type(), + case socket:ioctl(Sock, gifdstaddr, IfName) of + {ok, #{family := Fam, + addr := Addr}} -> + i("[~s] got (expected) (dest) socket address " + "for interface ~p (~w): " + "~n (~w) ~p", [Prefix, IfName, IfIdx, Fam, Addr]), + ok; + {ok, Crap} -> + %% Oups?! + i("<ERROR> [~s] got unexpected result for interface ~p (~w)" + "~n ~p", [Prefix, IfName, IfIdx, Crap]), + socket:close(Sock), + ?FAIL({unexpected_addr, Prefix, IfName, IfIdx, Crap}); + {error, eaddrnotavail = Reason} -> + i("[~s] got unexpected error for interface ~p (~w) => " + "SKIP interface" + "~n Reason: ~p", [Prefix, IfName, IfIdx, Reason]), + ignore; + {error, eperm = Reason} -> + i("[~s] got unexpected error for interface ~p (~w) => " + "SKIP interface" + "~n Reason: ~p", [Prefix, IfName, IfIdx, Reason]), + ignore; + {error, einval = Reason} when (OsFam =:= unix) andalso + ((OsName =:= darwin) orelse + (OsName =:= freebsd) orelse + (OsName =:= netbsd)) -> + i("[~s] got unexpected error for interface ~p (~w) => " + "SKIP interface" + "~n Reason: ~p", [Prefix, IfName, IfIdx, Reason]), + ignore; + {error, Reason} -> + i("<ERROR> got unexpected error for interface ~p (~w)" + "~n Reason: ~p", [IfName, IfIdx, Reason]), + socket:close(Sock), + ?FAIL({unexpected_failure, Prefix, IfName, IfIdx, Reason}) + end. + + + + +%% --- gifbrdaddr --- + +ioctl_get_gifbrdaddr(suite) -> + []; +ioctl_get_gifbrdaddr(doc) -> + []; +ioctl_get_gifbrdaddr(_Config) when is_list(_Config) -> + ?TT(?SECS(5)), + tc_try(?FUNCTION_NAME, + fun() -> + has_support_net_if_names(), + has_support_ioctl_gifbrdaddr() + end, + fun() -> + InitState = #{}, + ok = do_ioctl_get_gifbrdaddr(InitState) + end). + + +do_ioctl_get_gifbrdaddr(_State) -> + Domain = inet, + LSA = which_local_socket_addr(Domain), + + i("create and init listen stream:TCP socket"), + {ok, LSock} = socket:open(inet, stream, tcp), + ok = socket:bind(LSock, LSA#{port => 0}), + ok = socket:listen(LSock), + {ok, #{port := LPort}} = socket:sockname(LSock), + + i("create and init connection stream:TCP socket"), + {ok, CSock} = socket:open(inet, stream, tcp), + + i("attempt nowait connect"), + {select, {select_info, _Tag, SelectHandle} = _SelectInfo} = + socket:connect(CSock, LSA#{port => LPort}, nowait), + + i("attempt accept"), + {ok, ASock} = socket:accept(LSock), + + i("await connection ready"), + receive + {'$socket', CSock, select, SelectHandle} -> + ok + end, + + i("attmpt complete connection"), + ok = socket:connect(CSock), + + i("get if names"), + {ok, IfNames} = net:if_names(), + + i("try ioctl all if indexes: " + "~n ~p", [IfNames]), + %% This a *very* simple test... + %% ...just to check that we actually get an socket address + verify_gifbrdaddr(ASock, "accept", IfNames), + verify_gifbrdaddr(CSock, "connect", IfNames), + verify_gifbrdaddr(LSock, "listen", IfNames), + + i("close socket(s)"), + _ = socket:close(CSock), + _ = socket:close(ASock), + _ = socket:close(LSock), + + i("done"), + ok. + + +verify_gifbrdaddr(_Sock, _Prefix, []) -> + ok; +verify_gifbrdaddr(Sock, Prefix, [{IfIdx, IfName} | IfNames]) -> + i("[~s] attempt verify gifbrdaddr for interface ~s (~w)", + [Prefix, IfName, IfIdx]), + verify_gifbrdaddr(Sock, Prefix, IfIdx, IfName), + verify_gifbrdaddr(Sock, Prefix, IfNames). + +verify_gifbrdaddr(Sock, Prefix, IfIdx, IfName) -> + {OsFam, OsName} = os:type(), + case socket:ioctl(Sock, gifbrdaddr, IfName) of + {ok, #{family := Fam, + addr := Addr}} -> + i("[~s] got (expected) (broadcast) socket address for " + "interface ~p (~w): " + "~n (~w) ~p", [Prefix, IfName, IfIdx, Fam, Addr]), + ok; + {ok, Crap} -> + %% Oups?! + i("<ERROR> [~s] got unexpected result for interface ~p (~w)" + "~n ~p", [Prefix, IfName, IfIdx, Crap]), + socket:close(Sock), + ?FAIL({unexpected_addr, IfName, IfIdx, Crap}); + {error, eaddrnotavail = Reason} -> + i("[~s] got unexpected error for interface ~p (~w) => " + "SKIP interface" + "~n Reason: ~p", [Prefix, IfName, IfIdx, Reason]), + ignore; + {error, eperm = Reason} -> + i("[~s] got unexpected error for interface ~p (~w) => " + "SKIP interface" + "~n Reason: ~p", [Prefix, IfName, IfIdx, Reason]), + ignore; + {error, einval = Reason} when (OsFam =:= unix) andalso + ((OsName =:= darwin) orelse + (OsName =:= freebsd) orelse + (OsName =:= netbsd)) -> + i("[~s] got unexpected error for interface ~p (~w) => " + "SKIP interface" + "~n Reason: ~p", [Prefix, IfName, IfIdx, Reason]), + ignore; + {error, Reason} -> + i("<ERROR> [~s] got unexpected error for interface ~p (~w)" + "~n Reason: ~p", [Prefix, IfName, IfIdx, Reason]), + socket:close(Sock), + ?FAIL({unexpected_failure, IfName, IfIdx, Reason}) + end. + + + +%% --- gifnetmask --- + +ioctl_get_gifnetmask(suite) -> + []; +ioctl_get_gifnetmask(doc) -> + []; +ioctl_get_gifnetmask(_Config) when is_list(_Config) -> + ?TT(?SECS(5)), + tc_try(?FUNCTION_NAME, + fun() -> + has_support_net_if_names(), + has_support_ioctl_gifnetmask() + end, + fun() -> + InitState = #{}, + ok = do_ioctl_get_gifnetmask(InitState) + end). + + +do_ioctl_get_gifnetmask(_State) -> + i("create dummy stream:TCP socket"), + {ok, Sock} = socket:open(inet, stream, tcp), + + i("get if names"), + {ok, IfNames} = net:if_names(), + + i("try ioctl all if indexes: " + "~n ~p", [IfNames]), + %% This a *very* simple test... + %% ...just to check that we actually get an socket address + _ = [case socket:ioctl(Sock, gifnetmask, IfName) of + {ok, #{family := Fam, + addr := Addr}} -> + i("got (expected) (netmask) socket address for interface ~p (~w): " + "~n (~w) ~p", [IfName, IfIdx, Fam, Addr]), + ok; + {ok, Crap} -> + %% Oups?! + i("<ERROR> got unexpected result for interface ~p (~w)" + "~n ~p", [IfName, IfIdx, Crap]), + socket:close(Sock), + ?FAIL({unexpected_addr, IfName, IfIdx, Crap}); + {error, eaddrnotavail = Reason} -> + i("got unexpected error for interface ~p (~w) => SKIP interface" + "~n Reason: ~p", [IfName, IfIdx, Reason]), + ignore; + {error, eperm = Reason} -> + i("got unexpected error for interface ~p (~w) => " + "SKIP interface" + "~n Reason: ~p", [IfName, IfIdx, Reason]), + ignore; + {error, Reason} -> + i("<ERROR> got unexpected error for interface ~p (~w)" + "~n Reason: ~p", [IfName, IfIdx, Reason]), + socket:close(Sock), + ?FAIL({unexpected_failure, IfName, IfIdx, Reason}) + end || {IfIdx, IfName} <- IfNames], + + i("close dummy stream:TCP socket"), + ok = socket:close(Sock), + + i("done"), + ok. + + + +%% --- gifmtu --- + +ioctl_get_gifmtu(suite) -> + []; +ioctl_get_gifmtu(doc) -> + []; +ioctl_get_gifmtu(_Config) when is_list(_Config) -> + ?TT(?SECS(5)), + tc_try(?FUNCTION_NAME, + fun() -> + has_support_net_if_names(), + has_support_ioctl_gifmtu() + end, + fun() -> + InitState = #{}, + ok = do_ioctl_get_gifmtu(InitState) + end). + + +do_ioctl_get_gifmtu(_State) -> + i("create dummy stream:TCP socket"), + {ok, Sock} = socket:open(inet, stream, tcp), + + i("get if names"), + {ok, IfNames} = net:if_names(), + + i("try ioctl all if indexes: " + "~n ~p", [IfNames]), + %% This a *very* simple test... + %% ...just to check that we actually get an socket address + _ = [case socket:ioctl(Sock, gifmtu, IfName) of + {ok, MTU} when is_integer(MTU) -> + i("got (expected) MTU for interface ~p (~w): " + "~n ~p", [IfName, IfIdx, MTU]), + ok; + {ok, Crap} -> + %% Oups?! + i("<ERROR> got unexpected result for interface ~p (~w)" + "~n ~p", [IfName, IfIdx, Crap]), + socket:close(Sock), + ?FAIL({unexpected_mtu, IfName, IfIdx, Crap}); + {error, Reason} -> + i("<ERROR> got unexpected error for interface ~p (~w)" + "~n Reason: ~p", [IfName, IfIdx, Reason]), + socket:close(Sock), + ?FAIL({unexpected_failure, IfName, IfIdx, Reason}) + end || {IfIdx, IfName} <- IfNames], + + i("close dummy stream:TCP socket"), + ok = socket:close(Sock), + + i("done"), + ok. + + + +%% --- gifhwaddr --- + +ioctl_get_gifhwaddr(suite) -> + []; +ioctl_get_gifhwaddr(doc) -> + []; +ioctl_get_gifhwaddr(_Config) when is_list(_Config) -> + ?TT(?SECS(5)), + tc_try(?FUNCTION_NAME, + fun() -> + has_support_net_if_names(), + has_support_ioctl_gifhwaddr() + end, + fun() -> + InitState = #{}, + ok = do_ioctl_get_gifhwaddr(InitState) + end). + + +do_ioctl_get_gifhwaddr(_State) -> + i("create dummy dgram:UDP socket"), + {ok, Sock} = socket:open(inet, dgram, udp), + + i("get if names"), + {ok, IfNames} = net:if_names(), + + i("try ioctl all if indexes: " + "~n ~p", [IfNames]), + %% This a *very* simple test... + %% ...just to check that we actually get an socket address + _ = [case socket:ioctl(Sock, gifhwaddr, IfName) of + {ok, #{family := ArphdrFam, + addr := Addr}} when is_atom(ArphdrFam) -> + case erlang:atom_to_list(ArphdrFam) of + "arphrd_" ++ _ -> + i("got (expected) (HW) socket address for " + "interface ~p (~w): " + "~n (~w) ~p", [IfName, IfIdx, ArphdrFam, Addr]), + ok; + _ -> + i("<ERROR> got unexpected family for interface ~p (~w)" + "~n ~p", [IfName, IfIdx, ArphdrFam]), + socket:close(Sock), + ?FAIL({unexpected_family, IfName, IfIdx, ArphdrFam}) + end; + {ok, #{family := Fam, + addr := Addr}} when is_integer(Fam) -> + i("got (expected) socket address for interface ~p (~w): " + "~n (~w) ~p", [IfName, IfIdx, Fam, Addr]), + ok; + {ok, Crap} -> + %% Oups?! + i("<ERROR> got unexpected result for interface ~p (~w)" + "~n ~p", [IfName, IfIdx, Crap]), + socket:close(Sock), + ?FAIL({unexpected_addr, IfName, IfIdx, Crap}); + {error, Reason} -> + i("<ERROR> got unexpected error for interface ~p (~w)" + "~n Reason: ~p", [IfName, IfIdx, Reason]), + socket:close(Sock), + ?FAIL({unexpected_failure, IfName, IfIdx, Reason}) + end || {IfIdx, IfName} <- IfNames], + + i("close dummy dgram:UDP socket"), + ok = socket:close(Sock), + + i("done"), + ok. + + + +%% --- giftxqlen --- + +ioctl_get_giftxqlen(suite) -> + []; +ioctl_get_giftxqlen(doc) -> + []; +ioctl_get_giftxqlen(_Config) when is_list(_Config) -> + ?TT(?SECS(5)), + tc_try(?FUNCTION_NAME, + fun() -> + has_support_net_if_names(), + has_support_ioctl_giftxqlen() + end, + fun() -> + InitState = #{}, + ok = do_ioctl_get_giftxqlen(InitState) + end). + + +do_ioctl_get_giftxqlen(_State) -> + i("create dummy stream:TCP socket"), + {ok, Sock} = socket:open(inet, stream, tcp), + + i("get if names"), + {ok, IfNames} = net:if_names(), + + i("try ioctl all if indexes: " + "~n ~p", [IfNames]), + %% This a *very* simple test... + %% ...just to check that we actually get an socket address + _ = [case socket:ioctl(Sock, giftxqlen, IfName) of + {ok, QLen} when is_integer(QLen) -> + i("got (expected) TX QLen for interface ~p (~w): " + "~n ~p", [IfName, IfIdx, QLen]), + ok; + {ok, Crap} -> + %% Oups?! + i("<ERROR> got unexpected result for interface ~p (~w)" + "~n ~p", [IfName, IfIdx, Crap]), + socket:close(Sock), + ?FAIL({unexpected_mtu, IfName, IfIdx, Crap}); + {error, Reason} -> + i("<ERROR> got unexpected error for interface ~p (~w)" + "~n Reason: ~p", [IfName, IfIdx, Reason]), + socket:close(Sock), + ?FAIL({unexpected_failure, IfName, IfIdx, Reason}) + end || {IfIdx, IfName} <- IfNames], + + i("close dummy stream:TCP socket"), + ok = socket:close(Sock), + + i("done"), + ok. + + + +%% --- gifflags --- + +ioctl_get_gifflags(suite) -> + []; +ioctl_get_gifflags(doc) -> + []; +ioctl_get_gifflags(_Config) when is_list(_Config) -> + ?TT(?SECS(5)), + tc_try(?FUNCTION_NAME, + fun() -> + has_support_net_if_names(), + has_support_ioctl_gifflags() + end, + fun() -> + InitState = #{}, + ok = do_ioctl_get_gifflags(InitState) + end). + + +do_ioctl_get_gifflags(_State) -> + i("create dummy stream:TCP socket"), + {ok, Sock} = socket:open(inet, stream, tcp), + + i("get if names"), + {ok, IfNames} = net:if_names(), + + AllFlags = [Flag || {Flag, Supported} <- + socket:supports(ioctl_flags), Supported =:= true], + + i("try ioctl all if indexes: " + "~n ~p", [IfNames]), + %% This a *very* simple test... + %% ...just to check that we actually get an socket address + _ = [case socket:ioctl(Sock, gifflags, IfName) of + {ok, Flags} when is_list(Flags) -> + i("got flags for interface ~p (~w): " + "~n ~p", [IfName, IfIdx, Flags]), + case Flags -- AllFlags of + [] -> + i("flags accounted for"), + ok; + ExtraFlags -> + i("<ERROR> got superfluous flags for interface ~p (~w)" + "~n Received Flags: ~p" + "~n Superfluous Flags: ~p" + "~n All Supported Flags: ~p" + "~n", [IfName, IfIdx, Flags, ExtraFlags, AllFlags]), + socket:close(Sock), + ?FAIL({unexpected_superfluous_flags, + IfName, IfIdx, Flags, ExtraFlags, AllFlags}) + end; + {ok, Crap} -> + %% Oups?! + i("<ERROR> got unexpected result for interface ~p (~w)" + "~n ~p", [IfName, IfIdx, Crap]), + socket:close(Sock), + ?FAIL({unexpected_mtu, IfName, IfIdx, Crap}); + {error, Reason} -> + i("<ERROR> got unexpected error for interface ~p (~w)" + "~n Reason: ~p", [IfName, IfIdx, Reason]), + socket:close(Sock), + ?FAIL({unexpected_failure, IfName, IfIdx, Reason}) + end || {IfIdx, IfName} <- IfNames], + + i("close dummy stream:TCP socket"), + ok = socket:close(Sock), + + i("done"), + ok. + + + +%% --- gifmap --- + +ioctl_get_gifmap(suite) -> + []; +ioctl_get_gifmap(doc) -> + []; +ioctl_get_gifmap(_Config) when is_list(_Config) -> + ?TT(?SECS(5)), + tc_try(?FUNCTION_NAME, + fun() -> + has_support_net_if_names(), + has_support_ioctl_gifmap() + end, + fun() -> + InitState = #{}, + ok = do_ioctl_get_gifmap(InitState) + end). + + +do_ioctl_get_gifmap(_State) -> + i("create dummy stream:TCP socket"), + {ok, Sock} = socket:open(inet, stream, tcp), + + i("get if names"), + {ok, IfNames} = net:if_names(), + + i("try ioctl all if indexes: " + "~n ~p", [IfNames]), + %% This a *very* simple test... + %% ...just to check that we actually get an socket address + _ = [case socket:ioctl(Sock, gifmap, IfName) of + {ok, Map} when is_map(Map) -> + i("got (expected) map for interface ~p (~w): " + "~n ~p", [IfName, IfIdx, Map]), + ok; + {ok, Crap} -> + %% Oups?! + i("<ERROR> got unexpected result for interface ~p (~w)" + "~n ~p", [IfName, IfIdx, Crap]), + socket:close(Sock), + ?FAIL({unexpected_mtu, IfName, IfIdx, Crap}); + {error, Reason} -> + i("<ERROR> got unexpected error for interface ~p (~w)" + "~n Reason: ~p", [IfName, IfIdx, Reason]), + socket:close(Sock), + ?FAIL({unexpected_failure, IfName, IfIdx, Reason}) + end || {IfIdx, IfName} <- IfNames], + + i("close dummy stream:TCP socket"), + ok = socket:close(Sock), + + i("done"), + ok. + + + + %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %% This test case is intended to (simply) test that the counters %% for both read and write. @@ -48713,6 +49984,89 @@ has_support_sendfile() -> skip("Not supported: socket") end. + +has_support_net_if_names() -> + try net:if_names() of + {ok, N} when is_list(N) -> + ok; + _ -> + skip("Not supported: net:if_names()") + catch + error : notsup -> + skip("Not supported: net") + end. + +has_support_ioctl_requests() -> + try socket:supports(ioctl_requests) of + Reqs when is_list(Reqs) -> + ok; + _ -> + skip("Not supported: ioctl_requests") + catch + error : notsup -> + skip("Not supported: socket") + end. + +has_support_ioctl_gifconf() -> + has_support_ioctl_request(gifconf). + +has_support_ioctl_gifname() -> + has_support_ioctl_request(gifname). + +has_support_ioctl_gifindex() -> + has_support_ioctl_request(gifindex). + +has_support_ioctl_gifaddr() -> + has_support_ioctl_request(gifaddr). + +has_support_ioctl_gifdstaddr() -> + has_support_ioctl_request(gifdstaddr). + +has_support_ioctl_gifbrdaddr() -> + has_support_ioctl_request(gifbrdaddr). + +has_support_ioctl_gifnetmask() -> + has_support_ioctl_request(gifnetmask). + +has_support_ioctl_gifmtu() -> + has_support_ioctl_request(gifmtu). + +has_support_ioctl_gifhwaddr() -> + has_support_ioctl_request(gifhwaddr). + +has_support_ioctl_giftxqlen() -> + has_support_ioctl_request(giftxqlen). + +has_support_ioctl_gifflags() -> + has_support_ioctl_request(gifflags). + +has_support_ioctl_gifmap() -> + has_support_ioctl_request(gifmap). + +has_support_ioctl_request(Req) when is_atom(Req) -> + try socket:is_supported(ioctl_requests, Req) of + true -> + ok; + false -> + skip(f("Not supported: ioctl_request: ~w", [Req])) + catch + error : notsup -> + skip("Not supported: socket") + end. + + +%% has_support_ioctl_flags() -> +%% try socket:supports(ioctl_flags) of +%% Reqs when is_list(Reqs) -> +%% ok; +%% _ -> +%% skip("Not supported: ioctl_flags") +%% catch +%% error : notsup -> +%% skip("Not supported: socket") +%% end. + + %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% unlink_path(Path) -> diff --git a/lib/kernel/vsn.mk b/lib/kernel/vsn.mk index af7a614427..670e637a8d 100644 --- a/lib/kernel/vsn.mk +++ b/lib/kernel/vsn.mk @@ -1 +1 @@ -KERNEL_VSN = 8.1.2 +KERNEL_VSN = 8.1.3 diff --git a/lib/megaco/Makefile b/lib/megaco/Makefile index 624c4b0619..f66f75876b 100644 --- a/lib/megaco/Makefile +++ b/lib/megaco/Makefile @@ -115,7 +115,6 @@ include $(ERL_TOP)/make/otp_subdir.mk reconf: (cd $(ERL_TOP) && \ - ./otp_build autoconf && \ ./otp_build configure && \ cd $(ERL_TOP)/../libraries/megaco) diff --git a/lib/mnesia/src/mnesia.erl b/lib/mnesia/src/mnesia.erl index 028deccea4..0d139e7867 100644 --- a/lib/mnesia/src/mnesia.erl +++ b/lib/mnesia/src/mnesia.erl @@ -316,28 +316,30 @@ kill() -> ms() -> [ + mnesia_sup, + mnesia_kernel_sup, + mnesia_checkpoint_sup, + mnesia_snmp_sup, + mnesia_ext_sup, + mnesia, mnesia_app, mnesia_backup, mnesia_bup, mnesia_checkpoint, - mnesia_checkpoint_sup, mnesia_controller, mnesia_dumper, mnesia_loader, mnesia_frag, mnesia_frag_hash, mnesia_index, - mnesia_kernel_sup, mnesia_late_loader, mnesia_lib, mnesia_log, mnesia_registry, mnesia_schema, mnesia_snmp_hook, - mnesia_snmp_sup, mnesia_subscr, - mnesia_sup, mnesia_text, mnesia_tm, mnesia_recover, @@ -345,7 +347,6 @@ ms() -> %% Keep these last in the list, so %% mnesia_sup kills these last - mnesia_ext_sup, mnesia_monitor, mnesia_event ]. diff --git a/lib/mnesia/src/mnesia_checkpoint.erl b/lib/mnesia/src/mnesia_checkpoint.erl index 3273d3d27e..969af80368 100644 --- a/lib/mnesia/src/mnesia_checkpoint.erl +++ b/lib/mnesia/src/mnesia_checkpoint.erl @@ -579,7 +579,7 @@ call(Name, Msg) -> {'DOWN', Monitor, _, Pid, Reason} -> {error, {"Got exit", [Name, Reason]}}; {Name, Self, Reply} -> - erlang:demonitor(Monitor), + erlang:demonitor(Monitor, [flush]), Reply end; Error -> @@ -817,7 +817,6 @@ retainer_loop(Cp = #checkpoint_args{is_activated=false, name=Name}) -> {From, deactivate} -> do_stop(Cp), reply(From, Name, deactivated), - unlink(From), exit(shutdown); {From, get_checkpoint} -> @@ -920,7 +919,6 @@ retainer_loop(Cp = #checkpoint_args{name=Name}) -> {From, deactivate} -> do_stop(Cp), reply(From, Name, deactivated), - unlink(From), exit(shutdown); {_From, {mnesia_down, Node}} -> diff --git a/lib/mnesia/src/mnesia_sup.erl b/lib/mnesia/src/mnesia_sup.erl index fd5156d2a4..6deda768ec 100644 --- a/lib/mnesia/src/mnesia_sup.erl +++ b/lib/mnesia/src/mnesia_sup.erl @@ -88,13 +88,25 @@ add_event_handler() -> kill() -> Mnesia = [mnesia_fallback | mnesia:ms()], - Kill = fun(Name) -> try exit(whereis(Name), kill) catch _:_ -> ok end end, + Kill = fun + (mnesia_sup) -> + try %% Avoid crash reports + Sup = whereis(mnesia_sup), + {_,Dict} = process_info(Sup, dictionary), + [App] = proplists:get_value('$ancestors', Dict), + unlink(App), + exit(Sup, kill), + App ! {'EXIT', Sup, normal} + catch _:_ -> ok end; + (Name) -> + try exit(whereis(Name), kill) catch _:_ -> ok end + end, lists:foreach(Kill, Mnesia), lists:foreach(fun ensure_dead/1, Mnesia), timer:sleep(10), case lists:keymember(mnesia, 1, application:which_applications()) of - true -> kill(); - false -> ok + true -> kill(); + false -> ok end. ensure_dead(Name) -> diff --git a/lib/mnesia/test/Makefile b/lib/mnesia/test/Makefile index 2b60caf5ee..7cc6ebb6d1 100644 --- a/lib/mnesia/test/Makefile +++ b/lib/mnesia/test/Makefile @@ -100,6 +100,8 @@ EBIN = . tests debug opt: $(TARGET_FILES) +$(TARGET_FILES): $(HRL_FILES) + $(EBIN)/%.beam: $(DocExamplesDir)/%.erl $(ERLC) -bbeam $(ERL_COMPILE_FLAGS) -o$(EBIN) $< diff --git a/lib/mnesia/test/ext_test.erl b/lib/mnesia/test/ext_test.erl index cd39f4a6f1..a95c804486 100644 --- a/lib/mnesia/test/ext_test.erl +++ b/lib/mnesia/test/ext_test.erl @@ -70,15 +70,12 @@ semantics(_Alias, _) -> init_backend() -> ?DBG(init_backend), - ct:log("init_backend ~p", [?MODULE]), %% cheat and stuff a marker in mnesia_gvar K = backend_init_marker(), case try mnesia_lib:val(K) catch _:_ -> error end of error -> - ct:log("BACKEND marker will be inserted (~p)", [?MODULE]), mnesia_lib:set(K, true); Other -> - ct:log("BACKEND marker already present (~p)", [?MODULE]), error({backend_already_initialized, {?MODULE, Other}}) end, ok. @@ -88,7 +85,7 @@ backend_init_marker() -> add_aliases(_As) -> ?DBG(_As), - ct:log("add_aliases(~p)", [_As]), + %ct:log("add_aliases(~p)", [_As]), true = mnesia_lib:val(backend_init_marker()), ok. @@ -148,7 +145,7 @@ receiver_first_message(Sender, {first, Size}, _Alias, Tab) -> ?DBG({first,Size}), {Size, {Tab, Sender}}. -receive_data(Data, ext_ets, Name, _Sender, {Name, Tab, _Sender}=State) -> +receive_data(Data, ext_ets, Name, Sender, {Name, Tab, Sender}=State) -> ?DBG({Data,State}), true = ets:insert(Tab, Data), {more, State}; diff --git a/lib/mnesia/test/mnesia_evil_coverage_test.erl b/lib/mnesia/test/mnesia_evil_coverage_test.erl index f6f2bfd45e..5a664f3834 100644 --- a/lib/mnesia/test/mnesia_evil_coverage_test.erl +++ b/lib/mnesia/test/mnesia_evil_coverage_test.erl @@ -2572,8 +2572,8 @@ index_cleanup(Config) when is_list(Config) -> Tabs = [i_set, i_bag, i_oset], Add = fun(Tab) -> - Write = fun(Tab) -> - Recs = [{Tab, N, N rem 5} || N <- lists:seq(1,10)], + Write = fun(Table) -> + Recs = [{Table, N, N rem 5} || N <- lists:seq(1,10)], [ok = mnesia:write(Rec) || Rec <- Recs], Recs end, @@ -2582,8 +2582,8 @@ index_cleanup(Config) when is_list(Config) -> end, IRead = fun(Tab) -> - Read = fun(Tab) -> - [mnesia:index_read(Tab, N, val) || N <- lists:seq(0,4)] + Read = fun(Table) -> + [mnesia:index_read(Table, N, val) || N <- lists:seq(0,4)] end, {atomic, Recs} = mnesia:transaction(Read, [Tab]), lists:sort(lists:flatten(Recs)) diff --git a/lib/mnesia/test/mnesia_test_lib.hrl b/lib/mnesia/test/mnesia_test_lib.hrl index b8eeb5783f..77b5e543d2 100644 --- a/lib/mnesia/test/mnesia_test_lib.hrl +++ b/lib/mnesia/test/mnesia_test_lib.hrl @@ -46,51 +46,51 @@ end()). -define(match(ExpectedRes,Expr), - fun() -> - try Expr of - _AR_0 = ExpectedRes -> - ?verbose("ok, ~n Result as expected:~p~n",[_AR_0]), - {success,_AR_0}; - _AR_0 -> - ?error("Not Matching Actual result was:~n ~p~n",[_AR_0]), - {fail,_AR_0} - catch - exit:{aborted, _ER_1}:Stacktrace when - element(1, _ER_1) =:= node_not_running; - element(1, _ER_1) =:= bad_commit; - element(1, _ER_1) =:= cyclic -> - %% Need to re-raise these to restart transaction - erlang:raise(exit, {aborted, _ER_1}, Stacktrace); - exit:_AR_1:Stacktrace -> - case fun(_AR_EXIT_) -> {'EXIT', _AR_EXIT_} end(_AR_1) of - _AR_2 = ExpectedRes -> - ?verbose("ok, ~n Result as expected:~p~n",[_AR_2]), - {success,_AR_2}; - _AR_2 -> - ?error("Not Matching Actual result was:~n ~p~n ~p~n", - [_AR_2, Stacktrace]), - {fail,_AR_2} - end; - _T1_:_AR_1:Stacktrace -> - ?error("Not Matching Actual result was:~n ~p~n ~p~n", - [{_T1_,_AR_1}, Stacktrace]), - {fail,{_T1_,_AR_1}} - end - end()). + (_ = fun() -> + try Expr of + _AR_0 = ExpectedRes -> + ?verbose("ok, ~n Result as expected:~p~n",[_AR_0]), + {success,_AR_0}; + _AR_0 -> + ?error("Not Matching Actual result was:~n ~p~n",[_AR_0]), + {fail,_AR_0} + catch + exit:{aborted, _ER_1}:Stacktrace when + element(1, _ER_1) =:= node_not_running; + element(1, _ER_1) =:= bad_commit; + element(1, _ER_1) =:= cyclic -> + %% Need to re-raise these to restart transaction + erlang:raise(exit, {aborted, _ER_1}, Stacktrace); + exit:_AR_1:Stacktrace -> + case fun(_AR_EXIT_) -> {'EXIT', _AR_EXIT_} end(_AR_1) of + _AR_2 = ExpectedRes -> + ?verbose("ok, ~n Result as expected:~p~n",[_AR_2]), + {success,_AR_2}; + _AR_2 -> + ?error("Not Matching Actual result was:~n ~p~n ~p~n", + [_AR_2, Stacktrace]), + {fail,_AR_2} + end; + _T1_:_AR_1:Stacktrace -> + ?error("Not Matching Actual result was:~n ~p~n ~p~n", + [{_T1_,_AR_1}, Stacktrace]), + {fail,{_T1_,_AR_1}} + end + end())). -define(match_inverse(NotExpectedRes,Expr), - fun() -> - AcTuAlReS = (catch (Expr)), - case AcTuAlReS of - NotExpectedRes -> - ?error("Not matching Actual result was:~n ~p~n", - [AcTuAlReS]), - {fail,AcTuAlReS}; - _ -> - ?verbose("ok, ~n Result as expected: ~p~n",[AcTuAlReS]), - {success,AcTuAlReS} - end - end()). + (_ = fun() -> + AcTuAlReS = (catch (Expr)), + case AcTuAlReS of + NotExpectedRes -> + ?error("Not matching Actual result was:~n ~p~n", + [AcTuAlReS]), + {fail,AcTuAlReS}; + _ -> + ?verbose("ok, ~n Result as expected: ~p~n",[AcTuAlReS]), + {success,AcTuAlReS} + end + end())). -define(match_receive(ExpectedMsg), ?match(ExpectedMsg,mnesia_test_lib:pick_msg())). diff --git a/lib/mnesia/test/mt.erl b/lib/mnesia/test/mt.erl index 75ed856a95..e086bac439 100644 --- a/lib/mnesia/test/mt.erl +++ b/lib/mnesia/test/mt.erl @@ -39,7 +39,7 @@ read_config/0, write_config/1 % Config admin ]). --include("mnesia_test_lib.hrl"). +-compile({no_auto_import,[alias/1]}). %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %% Aliases for the (sub) test suites diff --git a/lib/observer/test/crashdump_viewer_SUITE.erl b/lib/observer/test/crashdump_viewer_SUITE.erl index fdc235a3a3..901600ae09 100644 --- a/lib/observer/test/crashdump_viewer_SUITE.erl +++ b/lib/observer/test/crashdump_viewer_SUITE.erl @@ -45,11 +45,18 @@ init_per_testcase(start_stop, Config) -> case os:type() of {unix,darwin} -> exit("Can not test on MacOSX"); + {unix,sunos} -> + exit("Skip on sunos, for now"); {unix, _} -> - io:format("DISPLAY ~s~n", [os:getenv("DISPLAY")]), + Display = os:getenv("DISPLAY"), + io:format("DISPLAY ~s~n", [Display]), case ct:get_config(xserver, none) of none -> ignore; - Server -> os:putenv("DISPLAY", Server) + Display -> ok; + Server -> + os:putenv("DISPLAY", Server), %% Might work if new node is spawned + io:format("Config sets other x-server than the DISPLAY\n" + "the DISPLAY variable must be set when starting erlang") end; _ -> ignore end, diff --git a/lib/observer/test/observer_SUITE.erl b/lib/observer/test/observer_SUITE.erl index 7d5e90d2ce..575f63b956 100644 --- a/lib/observer/test/observer_SUITE.erl +++ b/lib/observer/test/observer_SUITE.erl @@ -96,12 +96,19 @@ init_per_group(gui = Group, Config) -> {unix,darwin} -> ?P("init_per_group(~w) -> skip", [Group]), exit("Can not test on MacOSX"); + {unix,sunos} -> + exit("Skip on sunos, for now"); {unix, _} -> + Display = os:getenv("DISPLAY"), ?P("init_per_group(~w) -> DISPLAY ~s", - [Group, os:getenv("DISPLAY")]), + [Group, Display]), case ct:get_config(xserver, none) of none -> ignore; - Server -> os:putenv("DISPLAY", Server) + Display -> ok; + Server -> + os:putenv("DISPLAY", Server), %% Might work if new node is spawned + io:format("Config sets other x-server than the DISPLAY\n" + "the DISPLAY variable must be set when starting erlang") end; _ -> ignore end, diff --git a/lib/parsetools/doc/src/yecc.xml b/lib/parsetools/doc/src/yecc.xml index 67847e5cc1..ba431b3b87 100644 --- a/lib/parsetools/doc/src/yecc.xml +++ b/lib/parsetools/doc/src/yecc.xml @@ -189,13 +189,13 @@ text, and turns it into one or more lists of tokens. Each token should be a tuple containing information about syntactic category, position in the text (e.g. line number), and the - actual terminal symbol found in the text: <c>{Category, LineNumber, Symbol}</c>.</p> + actual terminal symbol found in the text: <c>{Category, Position, Symbol}</c>.</p> <p>If a terminal symbol is the only member of a category, and the symbol name is identical to the category name, the token format - may be <c>{Symbol, LineNumber}</c>.</p> + may be <c>{Symbol, Position}</c>.</p> <p>A list of tokens produced by the scanner should end with a special <c>end_of_input</c> tuple which the parser is looking - for. The format of this tuple should be <c>{Endsymbol, LastLineNumber}</c>, where <c>Endsymbol</c> is an identifier + for. The format of this tuple should be <c>{Endsymbol, EndPosition}</c>, where <c>Endsymbol</c> is an identifier that is distinguished from all the terminal and non-terminal categories of the syntax rules. The <c>Endsymbol</c> may be declared in the grammar file (see below).</p> @@ -408,7 +408,7 @@ myparser:parse(myscanner:scan(Inport)) </code> default file <c>lib/parsetools/include/yeccpre.hrl</c>.</p> <p>With the standard prologue, this call will return either <c>{ok, Result}</c>, where <c>Result</c> is a structure that the - Erlang code of the grammar file has built, or <c>{error, {Line_number, Module, Message}}</c> if there was a syntax error + Erlang code of the grammar file has built, or <c>{error, {Position, Module, Message}}</c> if there was a syntax error in the input.</p> <p><c>Message</c> is something which may be converted into a string by calling <c>Module:format_error(Message)</c> @@ -433,13 +433,13 @@ myparser:parse_and_scan({Mod, Tokenizer, Args}) </code> <p>The tokenizer used above has to be implemented so as to return one of the following:</p> <code type="none"> -{ok, Tokens, Endline} -{eof, Endline} -{error, Error_description, Endline} </code> +{ok, Tokens, EndPosition} +{eof, EndPosition} +{error, Error_description, EndPosition} </code> <p>This conforms to the format used by the scanner in the Erlang <c>io</c> library module.</p> - <p>If <c>{eof, Endline}</c> is returned immediately, the call to - <c>parse_and_scan/1</c> returns <c>{ok, eof}</c>. If <c>{eof, Endline}</c> is returned before the parser expects end of input, + <p>If <c>{eof, EndPosition}</c> is returned immediately, the call to + <c>parse_and_scan/1</c> returns <c>{ok, eof}</c>. If <c>{eof, EndPosition}</c> is returned before the parser expects end of input, <c>parse_and_scan/1</c> will, of course, return an error message (see above). Otherwise <c>{ok, Result}</c> is returned.</p> </section> @@ -541,7 +541,7 @@ line_of(Token) -> rules, and an error is thrown (and caught by the generated parser to produce an error message) when a test fails. The same effect can be achieved with a call to - <c>return_error(Error_line, Message_string)</c>, which is + <c>return_error(ErrorPosition, Message_string)</c>, which is defined in the <c>yeccpre.hrl</c> default header file.</p> </note> </section> diff --git a/lib/parsetools/include/yeccpre.hrl b/lib/parsetools/include/yeccpre.hrl index 6cfdb60078..e0b132c180 100644 --- a/lib/parsetools/include/yeccpre.hrl +++ b/lib/parsetools/include/yeccpre.hrl @@ -148,19 +148,28 @@ yecctoken_location(Token) -> end. -compile({nowarn_unused_function, yecctoken2string/1}). -yecctoken2string({atom, _, A}) -> io_lib:write_atom(A); -yecctoken2string({integer,_,N}) -> io_lib:write(N); -yecctoken2string({float,_,F}) -> io_lib:write(F); -yecctoken2string({char,_,C}) -> io_lib:write_char(C); -yecctoken2string({var,_,V}) -> io_lib:format("~s", [V]); -yecctoken2string({string,_,S}) -> io_lib:write_string(S); -yecctoken2string({reserved_symbol, _, A}) -> io_lib:write(A); -yecctoken2string({_Cat, _, Val}) -> io_lib:format("~tp", [Val]); -yecctoken2string({dot, _}) -> "'.'"; -yecctoken2string({'$end', _}) -> []; -yecctoken2string({Other, _}) when is_atom(Other) -> +yecctoken2string(Token) -> + try + yecctoken2string1(Token) + catch + _:_ -> + io_lib:format("~tp", [Token]) + end. + +-compile({nowarn_unused_function, yecctoken2string1/1}). +yecctoken2string1({atom, _, A}) -> io_lib:write_atom(A); +yecctoken2string1({integer,_,N}) -> io_lib:write(N); +yecctoken2string1({float,_,F}) -> io_lib:write(F); +yecctoken2string1({char,_,C}) -> io_lib:write_char(C); +yecctoken2string1({var,_,V}) -> io_lib:format("~s", [V]); +yecctoken2string1({string,_,S}) -> io_lib:write_string(S); +yecctoken2string1({reserved_symbol, _, A}) -> io_lib:write(A); +yecctoken2string1({_Cat, _, Val}) -> io_lib:format("~tp", [Val]); +yecctoken2string1({dot, _}) -> "'.'"; +yecctoken2string1({'$end', _}) -> []; +yecctoken2string1({Other, _}) when is_atom(Other) -> io_lib:write_atom(Other); -yecctoken2string(Other) -> +yecctoken2string1(Other) -> io_lib:format("~tp", [Other]). %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% diff --git a/lib/parsetools/test/yecc_SUITE.erl b/lib/parsetools/test/yecc_SUITE.erl index cfdb3e4458..83daa91535 100644 --- a/lib/parsetools/test/yecc_SUITE.erl +++ b/lib/parsetools/test/yecc_SUITE.erl @@ -1218,8 +1218,29 @@ yeccpre(Config) when is_list(Config) -> ok. ">>, default, + ok}, + + {error_3, <<" + Nonterminals statement. + Terminals keyword string. + Rootsymbol statement. + + statement -> keyword string. + + Erlang code. + + -export([t/0]). + + t() -> + %% Never crash in yecctoken_to_string/1 or its helpers, + %% even if when tokens are not in the format that erl_scan + %% produces. + {error, _} = parse([{string, 1, <<\"foo\">>}]), + ok. + ">>, + default, ok}], - + run(Config, Ts), ok. diff --git a/lib/sasl/src/release_handler_1.erl b/lib/sasl/src/release_handler_1.erl index ddaa5c7577..277655a28d 100644 --- a/lib/sasl/src/release_handler_1.erl +++ b/lib/sasl/src/release_handler_1.erl @@ -637,14 +637,19 @@ maybe_supervisor_which_children(Proc, Name, Pid) -> error(suspended_supervisor); running -> - case catch supervisor:which_children(Pid) of + try supervisor:which_children(Pid) of Res when is_list(Res) -> - Res; - Other -> + Res + catch + exit:Reason when Reason =/= timeout andalso + not (is_tuple(Reason) andalso + element(1,Reason) =:= nodedown) -> + []; + exit:Other -> error_logger:error_msg("release_handler: ~p~nerror during" " a which_children call to ~p (~w)." " [State: running] Exiting ... ~n", - [Other, Name, Pid]), + [{'EXIT',Other}, Name, Pid]), error(which_children_failed) end end. diff --git a/lib/snmp/doc/src/snmp.xml b/lib/snmp/doc/src/snmp.xml index 9219e58ab1..9a4e123cb2 100644 --- a/lib/snmp/doc/src/snmp.xml +++ b/lib/snmp/doc/src/snmp.xml @@ -4,7 +4,7 @@ <erlref> <header> <copyright> - <year>1996</year><year>2018</year> + <year>1996</year><year>2021</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -282,16 +282,16 @@ <name since="">passwd2localized_key(Alg, Passwd, EngineID) -> Key</name> <fsummary>Generates an localized key</fsummary> <type> - <v>Alg = algorithm()</v> - <v>algorithm() = md5 | sha</v> - <v>Passwd = string()</v> - <v>EngineID = string()</v> - <v>Key = list()</v> + <v>Alg = algorithm()</v> + <v>algorithm() = md5 | sha | sha224 | sha256 | sha384 | sha512</v> + <v>Passwd = string()</v> + <v>EngineID = string()</v> + <v>Key = list()</v> </type> <desc> <p>Generates a key that can be used as an authentication - or privacy key using MD5 och SHA. The key is - localized for EngineID.</p> + or privacy key using MD5, SHA, SHA224, SHA256, SHA384 or SHA512. + The key is localized for EngineID.</p> <marker id="octet_string_to_bits"></marker> </desc> diff --git a/lib/snmp/doc/src/snmp_agent_config_files.xml b/lib/snmp/doc/src/snmp_agent_config_files.xml index 8456b84952..a07d642687 100644 --- a/lib/snmp/doc/src/snmp_agent_config_files.xml +++ b/lib/snmp/doc/src/snmp_agent_config_files.xml @@ -4,7 +4,7 @@ <chapter> <header> <copyright> - <year>1997</year><year>2020</year> + <year>1997</year><year>2021</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -38,7 +38,8 @@ parameter. These files are read at start-up, and are used to initialize the SNMPv2-MIB or STANDARD-MIB, SNMP-FRAMEWORK-MIB, SNMP-MPD-MIB, SNMP-VIEW-BASED-ACM-MIB, SNMP-COMMUNITY-MIB, - SNMP-USER-BASED-SM-MIB, SNMP-TARGET-MIB and SNMP-NOTIFICATION-MIB + SNMP-USER-BASED-SM-MIB (adjusted according to SNMP-USM-HMAC-SHA2-MIB), + SNMP-TARGET-MIB and SNMP-NOTIFICATION-MIB (refer to the <seeguide marker="snmp_agent_funct_descr#management">Management of the Agent</seeguide> for a description of the MIBs). </p> @@ -347,7 +348,7 @@ file called <c>usm.conf</c>, which must be present if the agent is configured for SNMPv3. </p> <p>The corresponding table is <c>usmUserTable</c> in the - SNMP-USER-BASED-SM-MIB.</p> + SNMP-USER-BASED-SM-MIB (adjusted according to SNMP-USM-HMAC-SHA2-MIB).</p> <p>Each entry is a term:</p> <p><c>{EngineID, UserName, SecName, Clone, AuthP, AuthKeyC, OwnAuthKeyC, PrivP, PrivKeyC, OwnPrivKeyC, Public, AuthKey, PrivKey}.</c></p> <list type="bulleted"> @@ -365,7 +366,12 @@ </item> <item> <p><c>AuthP</c> is a <c>usmNoAuthProtocol</c>, - <c>usmHMACMD5AuthProtocol</c>, or <c>usmHMACSHAAuthProtocol</c>.</p> + <c>usmHMACMD5AuthProtocol</c>, + <c>usmHMACSHAAuthProtocol</c>, + <c>usmHMAC128SHA224AuthProtocol</c>, + <c>usmHMAC192SH256AuthProtocol</c>, + <c>usmHMAC256SHA384AuthProtocol</c> or + <c>usmHMAC384SHA512AuthProtocol</c>.</p> </item> <item> <p><c>AuthKeyC</c> is a string.</p> @@ -388,9 +394,28 @@ </item> <item> <p><c>AuthKey</c> is a list (of integer). This is the User's secret - localized authentication key. It is not visible in the MIB. The length - of this key needs to be 16 if <c>usmHMACMD5AuthProtocol</c> is used, - and 20 if <c>usmHMACSHAAuthProtocol</c> is used.</p> + localized authentication key. It is not visible in the MIB. + The length (number of octets) of this key needs to be:</p> + <list type="bulleted"> + <item> + <p>16 if <c>usmHMACMD5AuthProtocol</c>.</p> + </item> + <item> + <p>20 if <c>usmHMACSHAAuthProtocol</c>.</p> + </item> + <item> + <p>28 if <c>usmHMAC128SHA224AuthProtocol</c>.</p> + </item> + <item> + <p>32 if <c>usmHMAC192SHA256AuthProtocol</c>.</p> + </item> + <item> + <p>48 if <c>usmHMAC256SHA384AuthProtocol</c>.</p> + </item> + <item> + <p>64 if <c>usmHMAC384SHA512AuthProtocol</c>.</p> + </item> + </list> </item> <item> <p><c>PrivKey</c> is a list (of integer). This is the User's secret diff --git a/lib/snmp/doc/src/snmp_manager_config_files.xml b/lib/snmp/doc/src/snmp_manager_config_files.xml index 1db96afa78..fb741430c3 100644 --- a/lib/snmp/doc/src/snmp_manager_config_files.xml +++ b/lib/snmp/doc/src/snmp_manager_config_files.xml @@ -4,7 +4,7 @@ <chapter> <header> <copyright> - <year>2004</year><year>2016</year> + <year>2004</year><year>2021</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -245,7 +245,7 @@ <seeerl marker="snmpm#register_usm_user">register_usm_user</seeerl>. </p> <p>The corresponding table is <c>usmUserTable</c> in the - SNMP-USER-BASED-SM-MIB. + SNMP-USER-BASED-SM-MIB (adjusted according to SNMP-USM-HMAC-SHA2-MIB). </p> <p>Each entry is a term: </p> @@ -266,14 +266,38 @@ </item> <item> <p><c>AuthP</c> is a <c>usmNoAuthProtocol</c>, - <c>usmHMACMD5AuthProtocol</c> or <c>usmHMACSHAAuthProtocol</c>.</p> + <c>usmHMACMD5AuthProtocol</c>, + <c>usmHMACSHAAuthProtocol</c>, + <c>usmHMAC128SHA224AuthProtocol</c>, + <c>usmHMAC192SH256AuthProtocol</c>, + <c>usmHMAC256SHA384AuthProtocol</c> or + <c>usmHMAC384SHA512AuthProtocol</c>.</p> </item> <item> - <p><c>AuthKey</c> is a list (of integer). This is the User's - secret localized authentication key. It is not visible in the MIB. - The length of this key needs to be 16 if - <c>usmHMACMD5AuthProtocol</c> is used and 20 if - <c>usmHMACSHAAuthProtocol</c> is used.</p> + <p><c>AuthKey</c> is a list (of integer). + This is the User's secret localized authentication key. + It is not visible in the MIB. + The length (number of octets) of this key needs to be: </p> + <list type="bulleted"> + <item> + <p>16 if <c>usmHMACMD5AuthProtocol</c>.</p> + </item> + <item> + <p>20 if <c>usmHMACSHAAuthProtocol</c>.</p> + </item> + <item> + <p>28 if <c>usmHMAC128SHA224AuthProtocol</c>.</p> + </item> + <item> + <p>32 if <c>usmHMAC192SHA256AuthProtocol</c>.</p> + </item> + <item> + <p>48 if <c>usmHMAC256SHA384AuthProtocol</c>.</p> + </item> + <item> + <p>64 if <c>usmHMAC384SHA512AuthProtocol</c>.</p> + </item> + </list> </item> <item> <p><c>PrivP</c> is a <c>usmNoPrivProtocol</c>, diff --git a/lib/snmp/doc/src/snmp_user_based_sm_mib.xml b/lib/snmp/doc/src/snmp_user_based_sm_mib.xml index d487439160..779fd8ce3a 100644 --- a/lib/snmp/doc/src/snmp_user_based_sm_mib.xml +++ b/lib/snmp/doc/src/snmp_user_based_sm_mib.xml @@ -4,7 +4,7 @@ <erlref> <header> <copyright> - <year>1999</year><year>2020</year> + <year>1999</year><year>2021</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -36,9 +36,10 @@ <modulesummary>Instrumentation Functions for SNMP-USER-BASED-SM-MIB</modulesummary> <description> <p>The module <c>snmp_user_based_sm_mib</c> implements the instrumentation - functions for the SNMP-USER-BASED-SM-MIB, and functions for configuring - the database. - </p> + functions for the SNMP-USER-BASED-SM-MIB, and functions for configuring + the database. </p> + <p>Note that authentication has been extended according to RFC 7860 + (SNMP-USM-HMAC-SHA2-MIB). </p> <p>The configuration files are described in the SNMP User's Manual.</p> </description> <funcs> @@ -110,7 +111,7 @@ <v>Name = string()</v> <v>SecName = string()</v> <v>Clone = zeroDotZero | [integer()]</v> - <v>AuthP = usmNoAuthProtocol | usmHMACMD5AuthProtocol | usmHMACSHAAuthProtocol</v> + <v>AuthP = usmNoAuthProtocol | usmHMACMD5AuthProtocol | usmHMACSHAAuthProtocol | usmHMAC128SHA224AuthProtocol | usmHMAC192SH256AuthProtocol | usmHMAC256SHA384AuthProtocol | usmHMAC384SHA512AuthProtocol</v> <v>AuthKeyC = string()</v> <v>OwnAuthKeyC = string()</v> <v>PrivP = usmNoPrivProtocol | usmDESPrivProtocol</v> diff --git a/lib/snmp/doc/src/snmpa_conf.xml b/lib/snmp/doc/src/snmpa_conf.xml index 8a0df62939..b7ca747dc7 100644 --- a/lib/snmp/doc/src/snmpa_conf.xml +++ b/lib/snmp/doc/src/snmpa_conf.xml @@ -4,7 +4,7 @@ <erlref> <header> <copyright> - <year>2006</year><year>2020</year> + <year>2006</year><year>2021</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -737,7 +737,7 @@ word() = 0..65535 <v>UserName = string()</v> <v>SecName = string()</v> <v>Clone = zeroDotZero | [integer()]</v> - <v>AuthP = usmNoAuthProtocol | usmHMACMD5AuthProtocol, | usmHMACSHAAuthProtocol</v> + <v>AuthP = usmNoAuthProtocol | usmHMACMD5AuthProtocol | usmHMACSHAAuthProtocol | usmHMAC128SHA224AuthProtocol | usmHMAC192SH256AuthProtocol | usmHMAC256SHA384AuthProtocol | usmHMAC384SHA512AuthProtocol</v> <v>AuthKeyC = string()</v> <v>OwnAuthKeyC = string()</v> <v>PrivP = usmNoPrivProtocol | usmDESPrivProtocol | usmAesCfb128Protocol</v> diff --git a/lib/snmp/doc/src/snmpm.xml b/lib/snmp/doc/src/snmpm.xml index 7b755c457a..bb0826f2ce 100644 --- a/lib/snmp/doc/src/snmpm.xml +++ b/lib/snmp/doc/src/snmpm.xml @@ -4,7 +4,7 @@ <erlref> <header> <copyright> - <year>2004</year><year>2020</year> + <year>2004</year><year>2021</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -415,9 +415,13 @@ sec_level = noAuthNoPriv | authNoPriv | authPriv <p>The type of <c>Val</c> depends on <c>Item</c>: </p> <code type="none"><![CDATA[ sec_name = string() -auth = usmNoAuthProtocol | usmHMACMD5AuthProtocol | usmHMACSHAAuthProtocoltimeout +auth = usmNoAuthProtocol | usmHMACMD5AuthProtocol | usmHMACSHAAuthProtocol | usmHMAC128SHA224AuthProtocol | usmHMAC192SH256AuthProtocol | usmHMAC256SHA384AuthProtocol | usmHMAC384SHA512AuthProtocol auth_key = [integer()] (length 16 if auth = usmHMACMD5AuthProtocol, - length 20 if auth = usmHMACSHAAuthProtocol) + length 20 if auth = usmHMACSHAAuthProtocol, + length 28 if auth = usmHMAC128SHA224AuthProtocol, + length 32 if auth = usmHMAC192SHA256AuthProtocol, + length 48 if auth = usmHMAC256SHA384AuthProtocol, + length 64 if auth = usmHMAC384SHA512AuthProtocol) priv = usmNoPrivProtocol | usmDESPrivProtocol | usmAesCfb128Protocol priv_key = [integer()] (length is 16 if priv = usmDESPrivProtocol | usmAesCfb128Protocol). ]]></code> diff --git a/lib/snmp/doc/src/snmpm_conf.xml b/lib/snmp/doc/src/snmpm_conf.xml index e3c9322a32..5549bc3c2f 100644 --- a/lib/snmp/doc/src/snmpm_conf.xml +++ b/lib/snmp/doc/src/snmpm_conf.xml @@ -4,7 +4,7 @@ <erlref> <header> <copyright> - <year>2006</year><year>2020</year> + <year>2006</year><year>2021</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -287,7 +287,7 @@ <v>EngineID = string()</v> <v>UserName = string()</v> <v>SecName = string()</v> - <v>AuthP = usmNoAuthProtocol | usmHMACMD5AuthProtocol | usmHMACSHAAuthProtocol</v> + <v>AuthP = usmNoAuthProtocol | usmHMACMD5AuthProtocol | usmHMACSHAAuthProtocol | usmHMAC128SHA224AuthProtocol | usmHMAC192SH256AuthProtocol | usmHMAC256SHA384AuthProtocol | usmHMAC384SHA512AuthProtocol</v> <v>AuthKey = [integer()]</v> <v>PrivP = usmNoPrivProtocol | usmDESPrivProtocol | usmAesCfb128Protocol</v> <v>PrivKey = [integer()]</v> diff --git a/lib/snmp/mibs/Makefile.in b/lib/snmp/mibs/Makefile.in index a807e19913..6419009a6a 100644 --- a/lib/snmp/mibs/Makefile.in +++ b/lib/snmp/mibs/Makefile.in @@ -2,7 +2,7 @@ # %CopyrightBegin% # -# Copyright Ericsson AB 1996-2020. All Rights Reserved. +# Copyright Ericsson AB 1996-2021. All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. @@ -41,6 +41,11 @@ RELSYSDIR = $(RELEASE_PATH)/lib/snmp-$(VSN) # Common macros # ---------------------------------------------------- +OTP_MIBS = \ + OTP-REG \ + OTP-TC \ + OTP-SNMPEA-MIB + MIBS = \ RFC1213-MIB \ STANDARD-MIB \ @@ -56,9 +61,8 @@ MIBS = \ SNMP-USM-AES-MIB \ INET-ADDRESS-MIB \ TRANSPORT-ADDRESS-MIB \ - OTP-REG \ - OTP-TC \ - OTP-SNMPEA-MIB + SNMP-USM-HMAC-SHA2-MIB \ + $(OTP_MIBS) STD_v1_MIB_FILES = \ RFC1155-SMI.mib \ @@ -152,6 +156,9 @@ $(SNMP_BIN_TARGET_DIR)/SNMP-COMMUNITY-MIB.bin: \ $(SNMP_BIN_TARGET_DIR)/SNMP-USER-BASED-SM-MIB.bin: \ $(SNMP_BIN_TARGET_DIR)/SNMP-FRAMEWORK-MIB.bin +$(SNMP_BIN_TARGET_DIR)/SNMP-USM-HMAC-SHA2-MIB.bin: \ + $(SNMP_BIN_TARGET_DIR)/SNMP-FRAMEWORK-MIB.bin + $(SNMP_BIN_TARGET_DIR)/SNMP-VIEW-BASED-ACM-MIB.bin: \ $(SNMP_BIN_TARGET_DIR)/SNMP-FRAMEWORK-MIB.bin diff --git a/lib/snmp/mibs/SNMP-USM-HMAC-SHA2-MIB.mib b/lib/snmp/mibs/SNMP-USM-HMAC-SHA2-MIB.mib new file mode 100644 index 0000000000..1d6ae5c0c0 --- /dev/null +++ b/lib/snmp/mibs/SNMP-USM-HMAC-SHA2-MIB.mib @@ -0,0 +1,105 @@ +SNMP-USM-HMAC-SHA2-MIB DEFINITIONS ::= BEGIN + IMPORTS + MODULE-IDENTITY, OBJECT-IDENTITY, + mib-2 FROM SNMPv2-SMI -- RFC 2578 + snmpAuthProtocols FROM SNMP-FRAMEWORK-MIB; -- RFC 3411 + +snmpUsmHmacSha2MIB MODULE-IDENTITY + LAST-UPDATED "201604180000Z" -- 18 April 2016, midnight + ORGANIZATION "SNMPv3 Working Group" + CONTACT-INFO "WG email: OPSAWG@ietf.org + Subscribe: + https://www.ietf.org/mailman/listinfo/opsawg + Editor: Johannes Merkle + secunet Security Networks + Postal: Mergenthaler Allee 77 + D-65760 Eschborn + Germany + Phone: +49 20154543091 + Email: johannes.merkle@secunet.com + + Co-Editor: Manfred Lochter + Bundesamt fuer Sicherheit in der + Informationstechnik (BSI) + Postal: Postfach 200363 + D-53133 Bonn + Germany + Phone: +49 228 9582 5643 + Email: manfred.lochter@bsi.bund.de" + + DESCRIPTION + "Definitions of Object Identities needed for the use of + HMAC-SHA2 Authentication Protocols by SNMP's User-based Security + Model. + + Copyright (c) 2016 IETF Trust and the persons identified as + authors of the code. All rights reserved. + + Redistribution and use in source and binary forms, with or + without modification, is permitted pursuant to, and subject + to the license terms contained in, the Simplified BSD License + set forth in Section 4.c of the IETF Trust's Legal Provisions + Relating to IETF Documents + (http://trustee.ietf.org/license-info)." + + REVISION "201604180000Z" -- 18 April 2016, midnight + DESCRIPTION + "Version correcting the MODULE-IDENTITY value, + published as RFC 7860" + + REVISION "201510140000Z" -- 14 October 2015, midnight + DESCRIPTION + "Initial version, published as RFC 7630" + + ::= { mib-2 235 } + + +usmHMAC128SHA224AuthProtocol OBJECT-IDENTITY + STATUS current + DESCRIPTION "The Authentication Protocol + usmHMAC128SHA224AuthProtocol uses HMAC-SHA-224 and + truncates output to 128 bits." + REFERENCE "- Krawczyk, H., Bellare, M., and R. Canetti, + HMAC: Keyed-Hashing for Message Authentication, + RFC 2104. + - National Institute of Standards and Technology, + Secure Hash Standard (SHS), FIPS PUB 180-4, 2012." + ::= { snmpAuthProtocols 4 } + +usmHMAC192SHA256AuthProtocol OBJECT-IDENTITY + STATUS current + DESCRIPTION "The Authentication Protocol + usmHMAC192SHA256AuthProtocol uses HMAC-SHA-256 and + truncates output to 192 bits." + REFERENCE "- Krawczyk, H., Bellare, M., and R. Canetti, + HMAC: Keyed-Hashing for Message Authentication, + RFC 2104. + - National Institute of Standards and Technology, + Secure Hash Standard (SHS), FIPS PUB 180-4, 2012." + ::= { snmpAuthProtocols 5 } + +usmHMAC256SHA384AuthProtocol OBJECT-IDENTITY + STATUS current + DESCRIPTION "The Authentication Protocol + usmHMAC256SHA384AuthProtocol uses HMAC-SHA-384 and + truncates output to 256 bits." + REFERENCE "- Krawczyk, H., Bellare, M., and R. Canetti, + HMAC: Keyed-Hashing for Message Authentication, + RFC 2104. + - National Institute of Standards and Technology, + Secure Hash Standard (SHS), FIPS PUB 180-4, 2012." + ::= { snmpAuthProtocols 6 } + +usmHMAC384SHA512AuthProtocol OBJECT-IDENTITY + STATUS current + DESCRIPTION "The Authentication Protocol + usmHMAC384SHA512AuthProtocol uses HMAC-SHA-512 and + truncates output to 384 bits." + REFERENCE "- Krawczyk, H., Bellare, M., and R. Canetti, + HMAC: Keyed-Hashing for Message Authentication, + RFC 2104. + - National Institute of Standards and Technology, + Secure Hash Standard (SHS), FIPS PUB 180-4, 2012." + ::= { snmpAuthProtocols 7 } + +END diff --git a/lib/snmp/src/agent/depend.mk b/lib/snmp/src/agent/depend.mk index 49c7669e41..b5698f6247 100644 --- a/lib/snmp/src/agent/depend.mk +++ b/lib/snmp/src/agent/depend.mk @@ -2,7 +2,7 @@ # %CopyrightBegin% # -# Copyright Ericsson AB 2004-2019. All Rights Reserved. +# Copyright Ericsson AB 2004-2021. All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. @@ -266,8 +266,10 @@ $(EBIN)/snmp_target_mib.$(EMULATOR): \ $(EBIN)/snmp_user_based_sm_mib.$(EMULATOR): \ snmp_user_based_sm_mib.erl \ ../misc/snmp_verbosity.hrl \ + ../misc/snmp_usm.hrl \ ../../include/snmp_types.hrl \ ../../include/SNMP-USER-BASED-SM-MIB.hrl \ + ../../include/SNMP-USM-HMAC-SHA2-MIB.hrl \ ../../include/SNMPv2-TC.hrl $(EBIN)/snmp_view_based_acm_mib.$(EMULATOR): \ diff --git a/lib/snmp/src/agent/snmp_user_based_sm_mib.erl b/lib/snmp/src/agent/snmp_user_based_sm_mib.erl index 4842669fa4..0b94f4604a 100644 --- a/lib/snmp/src/agent/snmp_user_based_sm_mib.erl +++ b/lib/snmp/src/agent/snmp_user_based_sm_mib.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1999-2019. All Rights Reserved. +%% Copyright Ericsson AB 1999-2021. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -39,10 +39,12 @@ -export([check_usm/1]). -include("SNMP-USER-BASED-SM-MIB.hrl"). +-include("SNMP-USM-HMAC-SHA2-MIB.hrl"). -include("SNMP-USM-AES-MIB.hrl"). -include("SNMPv2-TC.hrl"). -include("snmpa_internal.hrl"). -include("snmp_types.hrl"). +-include("snmp_usm.hrl"). -define(VMODULE,"USM_MIB"). -include("snmp_verbosity.hrl"). @@ -55,7 +57,7 @@ %% Columns not accessible via SNMP -define(usmUserAuthKey, 14). -define(usmUserPrivKey, 15). --define(is_cloning, 16). +-define(is_cloning, 16). %%%----------------------------------------------------------------- @@ -167,9 +169,13 @@ check_usm({EngineID, Name, SecName, Clone, AuthP, AuthKeyC, OwnAuthKeyC, {ok, X} -> X end, - AuthProtoAlt = [{usmNoAuthProtocol, ?usmNoAuthProtocol}, - {usmHMACSHAAuthProtocol, ?usmHMACSHAAuthProtocol}, - {usmHMACMD5AuthProtocol, ?usmHMACMD5AuthProtocol}], + AuthProtoAlt = [{usmNoAuthProtocol, ?usmNoAuthProtocol}, + {usmHMACMD5AuthProtocol, ?usmHMACMD5AuthProtocol}, + {usmHMACSHAAuthProtocol, ?usmHMACSHAAuthProtocol}, + {usmHMAC128SHA224AuthProtocol, ?usmHMAC128SHA224AuthProtocol}, + {usmHMAC192SHA256AuthProtocol, ?usmHMAC192SHA256AuthProtocol}, + {usmHMAC256SHA384AuthProtocol, ?usmHMAC256SHA384AuthProtocol}, + {usmHMAC384SHA512AuthProtocol, ?usmHMAC384SHA512AuthProtocol}], {ok, AuthProto} = snmp_conf:check_atom(AuthP, AuthProtoAlt), snmp_conf:check_string(AuthKeyC), snmp_conf:check_string(OwnAuthKeyC), @@ -191,12 +197,16 @@ check_usm({EngineID, Name, SecName, Clone, AuthP, AuthKeyC, OwnAuthKeyC, check_usm(X) -> error({invalid_user, X}). -alen(usmNoAuthProtocol) -> any; -alen(usmHMACMD5AuthProtocol) -> 16; -alen(usmHMACSHAAuthProtocol) -> 20. +alen(usmNoAuthProtocol) -> any; +alen(usmHMACMD5AuthProtocol) -> 16; +alen(usmHMACSHAAuthProtocol) -> 20; +alen(usmHMAC128SHA224AuthProtocol) -> ?usmHMAC128SHA224AuthProtocol_secret_key_length; +alen(usmHMAC192SHA256AuthProtocol) -> ?usmHMAC192SHA256AuthProtocol_secret_key_length; +alen(usmHMAC256SHA384AuthProtocol) -> ?usmHMAC256SHA384AuthProtocol_secret_key_length; +alen(usmHMAC384SHA512AuthProtocol) -> ?usmHMAC384SHA512AuthProtocol_secret_key_length. -plen(usmNoPrivProtocol) -> any; -plen(usmDESPrivProtocol) -> 16; +plen(usmNoPrivProtocol) -> any; +plen(usmDESPrivProtocol) -> 16; plen(usmAesCfb128Protocol) -> 16. @@ -225,6 +235,26 @@ check_user(User) -> case is_crypto_supported(sha) of true -> ok; false -> exit({unsupported_crypto, sha_mac_96}) + end; + ?usmHMAC128SHA224AuthProtocol -> + case is_crypto_supported(sha224) of + true -> ok; + false -> exit({unsupported_crypto, sha224_hmac_sha_2}) + end; + ?usmHMAC192SHA256AuthProtocol -> + case is_crypto_supported(sha256) of + true -> ok; + false -> exit({unsupported_crypto, sha256_hmac_sha_2}) + end; + ?usmHMAC256SHA384AuthProtocol -> + case is_crypto_supported(sha384) of + true -> ok; + false -> exit({unsupported_crypto, sha384_hmac_sha_2}) + end; + ?usmHMAC384SHA512AuthProtocol -> + case is_crypto_supported(sha512) of + true -> ok; + false -> exit({unsupported_crypto, sha512_hmac_sha_2}) end end, case element(?usmUserPrivProtocol, User) of @@ -492,11 +522,19 @@ usmUserTable(print) -> Prefix, element(?usmUserCloneFrom, Row), Prefix, element(?usmUserAuthProtocol, Row), case element(?usmUserAuthProtocol, Row) of - ?usmNoAuthProtocol -> none; - ?usmHMACMD5AuthProtocol -> md5; - ?usmHMACSHAAuthProtocol -> sha; - md5 -> md5; - sha -> sha; + ?usmNoAuthProtocol -> none; + ?usmHMACMD5AuthProtocol -> md5; + ?usmHMACSHAAuthProtocol -> sha; + ?usmHMAC128SHA224AuthProtocol -> sha224; + ?usmHMAC192SHA256AuthProtocol -> sha256; + ?usmHMAC256SHA384AuthProtocol -> sha384; + ?usmHMAC384SHA512AuthProtocol -> sha512; + md5 -> md5; + sha -> sha; + sha224 -> sha224; + sha256 -> sha256; + sha384 -> sha384; + sha512 -> sha512; _ -> undefined end, Prefix, element(?usmUserAuthKeyChange, Row), @@ -637,12 +675,20 @@ verify_usmUserTable_col(?usmUserCloneFrom, Clone) -> end; verify_usmUserTable_col(?usmUserAuthProtocol, AuthP) -> case AuthP of - usmNoAuthProtocol -> ?usmNoAuthProtocol; - usmHMACSHAAuthProtocol -> ?usmHMACSHAAuthProtocol; - usmHMACMD5AuthProtocol -> ?usmHMACMD5AuthProtocol; - ?usmNoAuthProtocol -> ?usmNoAuthProtocol; - ?usmHMACSHAAuthProtocol -> ?usmHMACSHAAuthProtocol; - ?usmHMACMD5AuthProtocol -> ?usmHMACMD5AuthProtocol; + usmNoAuthProtocol -> ?usmNoAuthProtocol; + usmHMACSHAAuthProtocol -> ?usmHMACSHAAuthProtocol; + usmHMACMD5AuthProtocol -> ?usmHMACMD5AuthProtocol; + usmHMAC128SHA224AuthProtocol -> ?usmHMAC128SHA224AuthProtocol; + usmHMAC192SHA256AuthProtocol -> ?usmHMAC192SHA256AuthProtocol; + usmHMAC256SHA384AuthProtocol -> ?usmHMAC256SHA384AuthProtocol; + usmHMAC384SHA512AuthProtocol -> ?usmHMAC384SHA512AuthProtocol; + ?usmNoAuthProtocol -> ?usmNoAuthProtocol; + ?usmHMACSHAAuthProtocol -> ?usmHMACSHAAuthProtocol; + ?usmHMACMD5AuthProtocol -> ?usmHMACMD5AuthProtocol; + ?usmHMAC128SHA224AuthProtocol -> ?usmHMAC128SHA224AuthProtocol; + ?usmHMAC192SHA256AuthProtocol -> ?usmHMAC192SHA256AuthProtocol; + ?usmHMAC256SHA384AuthProtocol -> ?usmHMAC256SHA384AuthProtocol; + ?usmHMAC384SHA512AuthProtocol -> ?usmHMAC384SHA512AuthProtocol; _ -> wrongValue(?usmUserAuthProtocol) end; @@ -865,6 +911,14 @@ validate_auth_protocol(RowIndex, Cols) -> inconsistentValue(?usmUserAuthProtocol); ?usmHMACSHAAuthProtocol -> inconsistentValue(?usmUserAuthProtocol); + ?usmHMAC128SHA224AuthProtocol -> + inconsistentValue(?usmUserAuthProtocol); + ?usmHMAC192SHA256AuthProtocol -> + inconsistentValue(?usmUserAuthProtocol); + ?usmHMAC256SHA384AuthProtocol -> + inconsistentValue(?usmUserAuthProtocol); + ?usmHMAC384SHA512AuthProtocol -> + inconsistentValue(?usmUserAuthProtocol); _ -> wrongValue(?usmUserAuthProtocol) end; @@ -891,6 +945,30 @@ validate_auth_protocol(RowIndex, Cols) -> false -> wrongValue(?usmUserAuthProtocol) end; + ?usmHMAC128SHA224AuthProtocol -> + case is_crypto_supported(sha224) of + true -> ok; + false -> + wrongValue(?usmUserAuthProtocol) + end; + ?usmHMAC192SHA256AuthProtocol -> + case is_crypto_supported(sha256) of + true -> ok; + false -> + wrongValue(?usmUserAuthProtocol) + end; + ?usmHMAC256SHA384AuthProtocol -> + case is_crypto_supported(sha384) of + true -> ok; + false -> + wrongValue(?usmUserAuthProtocol) + end; + ?usmHMAC384SHA512AuthProtocol -> + case is_crypto_supported(sha512) of + true -> ok; + false -> + wrongValue(?usmUserAuthProtocol) + end; _ -> wrongValue(?usmUserAuthProtocol) end end; @@ -966,8 +1044,14 @@ validate_key_change(RowIndex, Cols, KeyChangeCol, Type) -> auth -> case get_auth_proto(RowIndex, Cols) of ?usmNoAuthProtocol -> ok; - ?usmHMACMD5AuthProtocol when Len =:= 32 -> ok; - ?usmHMACSHAAuthProtocol when Len =:= 40 -> ok; + ?usmHMACMD5AuthProtocol when Len =:= 32 -> ok; + ?usmHMACSHAAuthProtocol when Len =:= 40 -> ok; + + ?usmHMAC128SHA224AuthProtocol when Len =:= (2*?usmHMAC128SHA224AuthProtocol_secret_key_length) -> ok; + ?usmHMAC192SHA256AuthProtocol when Len =:= (2*?usmHMAC192SHA256AuthProtocol_secret_key_length) -> ok; + ?usmHMAC256SHA384AuthProtocol when Len =:= (2*?usmHMAC256SHA384AuthProtocol_secret_key_length) -> ok; + ?usmHMAC384SHA512AuthProtocol when Len =:= (2*?usmHMAC384SHA512AuthProtocol_secret_key_length) -> ok; + _ -> wrongValue(KeyChangeCol) end; priv -> @@ -1158,10 +1242,18 @@ get(Name, RowIndex, Cols) -> mk_key_change(Hash, OldKey, NewKey) -> KeyLen = length(NewKey), Alg = case Hash of - ?usmHMACMD5AuthProtocol -> md5; - ?usmHMACSHAAuthProtocol -> sha; - md5 -> md5; - sha -> sha + ?usmHMACMD5AuthProtocol -> md5; + ?usmHMACSHAAuthProtocol -> sha; + ?usmHMAC128SHA224AuthProtocol -> sha224; + ?usmHMAC192SHA256AuthProtocol -> sha256; + ?usmHMAC256SHA384AuthProtocol -> sha384; + ?usmHMAC384SHA512AuthProtocol -> sha512; + md5 -> md5; + sha -> sha; + sha224 -> sha224; + sha256 -> sha256; + sha384 -> sha384; + sha512 -> sha512 end, Random = mk_random(KeyLen), mk_key_change(Alg, OldKey, NewKey, KeyLen, Random). @@ -1181,10 +1273,18 @@ extract_new_key(?usmNoAuthProtocol, OldKey, _KeyChange) -> extract_new_key(Hash, OldKey, KeyChange) -> KeyLen = length(OldKey), Alg = case Hash of - ?usmHMACMD5AuthProtocol -> md5; - ?usmHMACSHAAuthProtocol -> sha; - md5 -> md5; - sha -> sha + ?usmHMACMD5AuthProtocol -> md5; + ?usmHMACSHAAuthProtocol -> sha; + ?usmHMAC128SHA224AuthProtocol -> sha224; + ?usmHMAC192SHA256AuthProtocol -> sha256; + ?usmHMAC256SHA384AuthProtocol -> sha384; + ?usmHMAC384SHA512AuthProtocol -> sha512; + md5 -> md5; + sha -> sha; + sha224 -> sha224; + sha256 -> sha256; + sha384 -> sha384; + sha512 -> sha512 end, {Random, Delta} = split(KeyLen, KeyChange, []), Digest = lists:sublist(binary_to_list(crypto:hash(Alg, OldKey++Random)), KeyLen), diff --git a/lib/snmp/src/agent/snmpa_conf.erl b/lib/snmp/src/agent/snmpa_conf.erl index c2e9d4025a..f5c9c3a87f 100644 --- a/lib/snmp/src/agent/snmpa_conf.erl +++ b/lib/snmp/src/agent/snmpa_conf.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2006-2019. All Rights Reserved. +%% Copyright Ericsson AB 2006-2021. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -94,7 +94,11 @@ Clone :: zeroDotZero | [non_neg_integer()], AuthP :: usmNoAuthProtocol | usmHMACMD5AuthProtocol | - usmHMACSHAAuthProtocol, + usmHMACSHAAuthProtocol | + usmHMAC128SHA224AuthProtocol | + usmHMAC192SHA256AuthProtocol | + usmHMAC256SHA384AuthProtocol | + usmHMAC384SHA512AuthProtocol, AuthKeyC :: string(), OwnAuthKeyC :: string(), PrivP :: usmNoPrivProtocol | @@ -105,6 +109,10 @@ Public :: string(), %% Size 16 for usmHMACMD5AuthProtocol %% Size 20 for usmHMACSHAAuthProtocol + %% Size 28 for usmHMAC128SHA224AuthProtocol + %% Size 32 for usmHMAC192SHA256AuthProtocol + %% Size 48 for usmHMAC256SHA384AuthProtocol + %% Size 64 for usmHMAC384SHA512AuthProtocol AuthKey :: [non_neg_integer()], %% Size 16 for usmDESPrivProtocol | usmAesCfb128Protocol PrivKey :: [non_neg_integer()] diff --git a/lib/snmp/src/agent/snmpa_usm.erl b/lib/snmp/src/agent/snmpa_usm.erl index 3c3a3c7c45..c65fcf2f8c 100644 --- a/lib/snmp/src/agent/snmpa_usm.erl +++ b/lib/snmp/src/agent/snmpa_usm.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1999-2020. All Rights Reserved. +%% Copyright Ericsson AB 1999-2021. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -95,14 +95,15 @@ process_incoming_msg(Packet, Data, SecParams, SecLevel, LocalEngineID) -> #usmSecurityParameters{msgAuthoritativeEngineID = MsgAuthEngineID, msgUserName = MsgUserName} -> ?vlog("process_incoming_msg -> USM security parms: " - "~n msgAuthEngineID: ~w" - "~n userName: ~p", [MsgAuthEngineID, MsgUserName]), + "~n msgAuthEngineID: ~w" + "~n userName: ~p", [MsgAuthEngineID, MsgUserName]), %% 3.2.3 ?vtrace("process_incoming_msg -> check engine id: 3.2.3",[]), case snmp_user_based_sm_mib:is_engine_id_known(MsgAuthEngineID) of true -> ok; false -> + ?vlog("process_incoming_msg -> engine id *not* known"), SecData1 = [MsgUserName], error(usmStatsUnknownEngineIDs, ?usmStatsUnknownEngineIDs_instance, %% OTP-3542 @@ -116,29 +117,30 @@ process_incoming_msg(Packet, Data, SecParams, SecLevel, LocalEngineID) -> User when element(?usmUserStatus, User) =:= ?'RowStatus_active' -> User; {_, Name,_,_,_,_,_,_,_,_,_,_,_, RowStatus,_,_} -> - ?vdebug("process_incoming_msg -> " - "found user ~p with wrong row status: ~p", - [Name, RowStatus]), + ?vlog("process_incoming_msg -> " + "found user ~p with wrong row status: ~p", + [Name, RowStatus]), SecData2 = [MsgUserName], error(usmStatsUnknownUserNames, ?usmStatsUnknownUserNames_instance, %% OTP-3542 undefined, [{sec_data, SecData2}]); _ -> % undefined or not active user + ?vlog("process_incoming_msg -> unknown user"), SecData2 = [MsgUserName], error(usmStatsUnknownUserNames, ?usmStatsUnknownUserNames_instance, %% OTP-3542 undefined, [{sec_data, SecData2}]) end, SecName = element(?usmUserSecurityName, UsmUser), - ?vtrace("process_incoming_msg -> securityName: ~p",[SecName]), + ?vtrace("process_incoming_msg -> securityName: ~p", [SecName]), %% 3.2.5 - implicit in following checks %% 3.2.6 - 3.2.7 ?vtrace("process_incoming_msg -> " "authenticate incoming: 3.2.5 - 3.2.7" - "~n ~p",[UsmUser]), - DiscoOrPlain = authenticate_incoming(Packet, - UsmSecParams, UsmUser, - SecLevel, LocalEngineID), + "~n USM User: ~p", [UsmUser]), + DiscoOrPlain = authenticate_incoming(Packet, + UsmSecParams, UsmUser, + SecLevel, LocalEngineID), %% 3.2.8 ?vtrace("process_incoming_msg -> " "decrypt scoped data: 3.2.8",[]), @@ -183,7 +185,8 @@ process_discovery_msg(MsgAuthEngineID, Data, SecLevel) -> authenticate_incoming(Packet, UsmSecParams, UsmUser, SecLevel, LocalEngineID) -> %% 3.2.6 - ?vtrace("authenticate_incoming -> 3.2.6", []), + ?vtrace("authenticate_incoming -> 3.2.6" + "~n SecLevel: ~p", [SecLevel]), AuthProtocol = element(?usmUserAuthProtocol, UsmUser), #usmSecurityParameters{msgAuthoritativeEngineID = MsgAuthEngineID, msgAuthoritativeEngineBoots = MsgAuthEngineBoots, @@ -191,12 +194,13 @@ authenticate_incoming(Packet, UsmSecParams, UsmUser, SecLevel, msgAuthenticationParameters = MsgAuthParams} = UsmSecParams, ?vtrace("authenticate_incoming -> Sec params: " - "~n MsgAuthEngineID: ~w" - "~n MsgAuthEngineBoots: ~p" - "~n MsgAuthEngineTime: ~p", + "~n MsgAuthEngineID: ~w" + "~n MsgAuthEngineBoots: ~p" + "~n MsgAuthEngineTime: ~p", [MsgAuthEngineID, MsgAuthEngineBoots, MsgAuthEngineTime]), case snmp_misc:is_auth(SecLevel) of true -> + ?vtrace("authenticate_incoming -> authenticate"), SecName = element(?usmUserSecurityName, UsmUser), case is_auth(AuthProtocol, element(?usmUserAuthKey, UsmUser), @@ -218,6 +222,7 @@ authenticate_incoming(Packet, UsmSecParams, UsmUser, SecLevel, end; false -> % noAuth + ?vtrace("authenticate_incoming -> don't authenticate"), plain end. @@ -262,19 +267,18 @@ authoritative(SecName, MsgAuthEngineBoots, MsgAuthEngineTime, LocalEngineID) -> non_authoritative(SecName, MsgAuthEngineID, MsgAuthEngineBoots, MsgAuthEngineTime) -> ?vtrace("non_authoritative -> entry with" - "~n SecName: ~p" - "~n MsgAuthEngineID: ~p" - "~n MsgAuthEngineBoots: ~p" - "~n MsgAuthEngineTime: ~p", - [SecName, - MsgAuthEngineID, MsgAuthEngineBoots, MsgAuthEngineTime]), + "~n SecName: ~p" + "~n MsgAuthEngineID: ~p" + "~n MsgAuthEngineBoots: ~p" + "~n MsgAuthEngineTime: ~p", + [SecName, MsgAuthEngineID, MsgAuthEngineBoots, MsgAuthEngineTime]), SnmpEngineBoots = get_engine_boots(MsgAuthEngineID), SnmpEngineTime = get_engine_time(MsgAuthEngineID), LatestRecvTime = get_engine_latest_time(MsgAuthEngineID), ?vtrace("non_authoritative -> " - "~n SnmpEngineBoots: ~p" - "~n SnmpEngineTime: ~p" - "~n LatestRecvTime: ~p", + "~n SnmpEngineBoots: ~p" + "~n SnmpEngineTime: ~p" + "~n LatestRecvTime: ~p", [SnmpEngineBoots, SnmpEngineTime, LatestRecvTime]), UpdateLCD = if @@ -311,11 +315,11 @@ non_authoritative(SecName, case InTimeWindow of false -> ?vinfo("NOT in time window: " - "~n SecName: ~p" - "~n SnmpEngineBoots: ~p" - "~n MsgAuthEngineBoots: ~p" - "~n SnmpEngineTime: ~p" - "~n MsgAuthEngineTime: ~p", + "~n SecName: ~p" + "~n SnmpEngineBoots: ~p" + "~n MsgAuthEngineBoots: ~p" + "~n SnmpEngineTime: ~p" + "~n MsgAuthEngineTime: ~p", [SecName, SnmpEngineBoots, MsgAuthEngineBoots, SnmpEngineTime, MsgAuthEngineTime]), @@ -357,7 +361,11 @@ is_auth(AuthProtocol, AuthKey, AuthParams, Packet, SecName, (MsgAuthEngineTime =:= 0) andalso (TermDiscoEnabled =:= true) andalso (TermDiscoStage2 =:= discovery)) -> %% 3.2.7a - ?vtrace("is_auth -> terminating discovery stage 2 - discovery",[]), + ?vtrace("is_auth -> terminating discovery stage 2 - discovery:" + "~n Local Boots: ~p" + "~n Local Time: ~p", + [get_local_engine_boots(LocalEngineID), + get_local_engine_time(LocalEngineID)]), discovery; SnmpEngineID when ((MsgAuthEngineBoots =:= 0) andalso (MsgAuthEngineTime =:= 0) andalso @@ -444,30 +452,35 @@ generate_outgoing_msg(Message, SecEngineID, SecName, SecData, SecLevel, LocalEngineID) -> %% 3.1.1 ?vtrace("generate_outgoing_msg -> [3.1.1] entry with" - "~n SecEngineID: ~p" - "~n SecName: ~p" - "~n SecLevel: ~w" - "~n LocalEngineID: ~p", + "~n SecEngineID: ~p" + "~n SecName: ~p" + "~n SecLevel: ~w" + "~n LocalEngineID: ~p", [SecEngineID, SecName, SecLevel, LocalEngineID]), {UserName, AuthProtocol, PrivProtocol, AuthKey, PrivKey} = case SecData of [] -> % 3.1.1b %% Not a response - read from LCD + ?vtrace("generate_outgoing_msh -> [3.1.1b] get user from sec name"), case snmp_user_based_sm_mib:get_user_from_security_name( SecEngineID, SecName) of User when element(?usmUserStatus, User) =:= ?'RowStatus_active' -> - {element(?usmUserName, User), + {element(?usmUserName, User), element(?usmUserAuthProtocol, User), element(?usmUserPrivProtocol, User), - element(?usmUserAuthKey, User), - element(?usmUserPrivKey, User)}; + element(?usmUserAuthKey, User), + element(?usmUserPrivKey, User)}; {_, Name,_,_,_,_,_,_,_,_,_,_,_, RowStatus,_,_} -> ?vdebug("generate_outgoing_msg -> " "found not active user ~p: ~p", [Name, RowStatus]), error(unknownSecurityName); - _ -> + _X -> + ?vdebug("generate_outgoing_msg -> unexpected" + "~n SecEngineID: ~p" + "~n SecName: ~p" + "~n ~p", [SecEngineID, SecName, _X]), error(unknownSecurityName) end; [MsgUserName] -> @@ -544,6 +557,7 @@ generate_discovery_msg(Message, _ -> %% Discovery step 2 + ?vtrace("generate_discovery_msg -> step 2 - get user from sec name"), case snmp_user_based_sm_mib:get_user_from_security_name( SecEngineID, SecName) of User when element(?usmUserStatus, User) =:= @@ -558,7 +572,11 @@ generate_discovery_msg(Message, "found user ~p with wrong row status: ~p", [Name, RowStatus]), error(unknownSecurityName); - _ -> + _X -> + ?vdebug("generate_discovery_msg -> unexpected" + "~n SecEngineID: ~p" + "~n SecName: ~p" + "~n ~p", [SecEngineID, SecName, _X]), error(unknownSecurityName) end end, diff --git a/lib/snmp/src/manager/depend.mk b/lib/snmp/src/manager/depend.mk index fd42ab13fa..4967e1b8d3 100644 --- a/lib/snmp/src/manager/depend.mk +++ b/lib/snmp/src/manager/depend.mk @@ -2,7 +2,7 @@ # %CopyrightBegin% # -# Copyright Ericsson AB 2004-2019. All Rights Reserved. +# Copyright Ericsson AB 2004-2021. All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. @@ -30,6 +30,7 @@ $(EBIN)/snmpm.$(EMULATOR): \ $(EBIN)/snmpm_config.$(EMULATOR): \ snmpm_config.erl \ ../../include/snmp_types.hrl \ + ../misc/snmp_usm.hrl \ ../misc/snmp_verbosity.hrl $(EBIN)/snmpm_mpd.$(EMULATOR): \ diff --git a/lib/snmp/src/manager/snmpm_config.erl b/lib/snmp/src/manager/snmpm_config.erl index 356ba44b08..8e3dc3be6e 100644 --- a/lib/snmp/src/manager/snmpm_config.erl +++ b/lib/snmp/src/manager/snmpm_config.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2004-2020. All Rights Reserved. +%% Copyright Ericsson AB 2004-2021. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -121,6 +121,7 @@ -include_lib("kernel/include/file.hrl"). -include("snmp_types.hrl"). -include("snmpm_internal.hrl"). +-include("snmp_usm.hrl"). -include("snmpm_usm.hrl"). -include("snmp_debug.hrl"). -include("snmp_verbosity.hrl"). @@ -712,7 +713,8 @@ get_usm_etime(SnmpEngineID) -> Key = {etime, SnmpEngineID}, case get_usm_cache(Key) of {ok, Diff} -> - {ok, snmp_misc:now(sec) - Diff}; + ETime = snmp_misc:now(sec) - Diff, + {ok, ETime}; _ -> {ok, 0} end. @@ -2168,6 +2170,7 @@ verify_usm_user(AuthP, AuthKey, PrivP, PrivKey) -> verify_usm_user_priv(PrivP, PrivKey), ok. + verify_usm_user_auth(usmNoAuthProtocol, AuthKey) -> case (catch snmp_conf:check_string(AuthKey, any)) of ok -> @@ -2175,6 +2178,7 @@ verify_usm_user_auth(usmNoAuthProtocol, AuthKey) -> _ -> error({invalid_auth_key, usmNoAuthProtocol}) end; + verify_usm_user_auth(usmHMACMD5AuthProtocol, AuthKey) when is_list(AuthKey) andalso (length(AuthKey) =:= 16) -> case is_crypto_supported(md5) of @@ -2193,6 +2197,7 @@ verify_usm_user_auth(usmHMACMD5AuthProtocol, AuthKey) when is_list(AuthKey) -> error({invalid_auth_key, usmHMACMD5AuthProtocol, Len}); verify_usm_user_auth(usmHMACMD5AuthProtocol, _AuthKey) -> error({invalid_auth_key, usmHMACMD5AuthProtocol}); + verify_usm_user_auth(usmHMACSHAAuthProtocol, AuthKey) when is_list(AuthKey) andalso (length(AuthKey) =:= 20) -> case is_crypto_supported(sha) of @@ -2211,9 +2216,91 @@ verify_usm_user_auth(usmHMACSHAAuthProtocol, AuthKey) when is_list(AuthKey) -> error({invalid_auth_key, usmHMACSHAAuthProtocol, Len}); verify_usm_user_auth(usmHMACSHAAuthProtocol, _AuthKey) -> error({invalid_auth_key, usmHMACSHAAuthProtocol}); + +verify_usm_user_auth(usmHMAC128SHA224AuthProtocol, AuthKey) + when is_list(AuthKey) andalso + (length(AuthKey) =:= ?usmHMAC128SHA224AuthProtocol_secret_key_length) -> + case is_crypto_supported(sha224) of + true -> + case snmp_conf:all_integer(AuthKey) of + true -> + ok; + _ -> + error({invalid_auth_key, usmHMAC128SHA224AuthProtocol}) + end; + false -> + error({unsupported_crypto, sha224}) + end; +verify_usm_user_auth(usmHMAC128SHA224AuthProtocol, AuthKey) when is_list(AuthKey) -> + Len = length(AuthKey), + error({invalid_auth_key, usmHMAC128SHA224AuthProtocol, Len}); +verify_usm_user_auth(usmHMAC128SHA224AuthProtocol, _AuthKey) -> + error({invalid_auth_key, usmHMAC128SHA224AuthProtocol}); + +verify_usm_user_auth(usmHMAC192SHA256AuthProtocol, AuthKey) + when is_list(AuthKey) andalso + (length(AuthKey) =:= ?usmHMAC192SHA256AuthProtocol_secret_key_length) -> + case is_crypto_supported(sha256) of + true -> + case snmp_conf:all_integer(AuthKey) of + true -> + ok; + _ -> + error({invalid_auth_key, usmHMAC192SHA256AuthProtocol}) + end; + false -> + error({unsupported_crypto, sha256}) + end; +verify_usm_user_auth(usmHMAC192SHA256AuthProtocol, AuthKey) when is_list(AuthKey) -> + Len = length(AuthKey), + error({invalid_auth_key, usmHMAC192SHA256AuthProtocol, Len}); +verify_usm_user_auth(usmHMAC192SHA256AuthProtocol, _AuthKey) -> + error({invalid_auth_key, usmHMAC192SHA256AuthProtocol}); + +verify_usm_user_auth(usmHMAC256SHA384AuthProtocol, AuthKey) + when is_list(AuthKey) andalso + (length(AuthKey) =:= ?usmHMAC256SHA384AuthProtocol_secret_key_length) -> + case is_crypto_supported(sha384) of + true -> + case snmp_conf:all_integer(AuthKey) of + true -> + ok; + _ -> + error({invalid_auth_key, usmHMAC256SHA384AuthProtocol}) + end; + false -> + error({unsupported_crypto, sha384}) + end; +verify_usm_user_auth(usmHMAC256SHA384AuthProtocol, AuthKey) when is_list(AuthKey) -> + Len = length(AuthKey), + error({invalid_auth_key, usmHMAC256SHA384AuthProtocol, Len}); +verify_usm_user_auth(usmHMAC256SHA384AuthProtocol, _AuthKey) -> + error({invalid_auth_key, usmHMAC256SHA384AuthProtocol}); + +verify_usm_user_auth(usmHMAC384SHA512AuthProtocol, AuthKey) + when is_list(AuthKey) andalso + (length(AuthKey) =:= ?usmHMAC384SHA512AuthProtocol_secret_key_length) -> + case is_crypto_supported(sha512) of + true -> + case snmp_conf:all_integer(AuthKey) of + true -> + ok; + _ -> + error({invalid_auth_key, usmHMAC384SHA512AuthProtocol}) + end; + false -> + error({unsupported_crypto, sha512}) + end; +verify_usm_user_auth(usmHMAC384SHA512AuthProtocol, AuthKey) when is_list(AuthKey) -> + Len = length(AuthKey), + error({invalid_auth_key, usmHMAC384SHA512AuthProtocol, Len}); +verify_usm_user_auth(usmHMAC384SHA512AuthProtocol, _AuthKey) -> + error({invalid_auth_key, usmHMAC384SHA512AuthProtocol}); + verify_usm_user_auth(AuthP, _AuthKey) -> error({invalid_auth_protocol, AuthP}). + verify_usm_user_priv(usmNoPrivProtocol, PrivKey) -> case (catch snmp_conf:check_string(PrivKey, any)) of ok -> @@ -3037,13 +3124,19 @@ do_update_usm_user_info(Key, User, sec_name, Val) -> %% end; ok = verify_usm_user_sec_name(Val), do_update_usm_user_info(Key, User#usm_user{sec_name = Val}); + do_update_usm_user_info(Key, User, auth, Val) when (Val =:= usmNoAuthProtocol) orelse (Val =:= usmHMACMD5AuthProtocol) orelse - (Val =:= usmHMACSHAAuthProtocol) -> + (Val =:= usmHMACSHAAuthProtocol) orelse + (Val =:= usmHMAC128SHA224AuthProtocol) orelse + (Val =:= usmHMAC192SHA256AuthProtocol) orelse + (Val =:= usmHMAC256SHA384AuthProtocol) orelse + (Val =:= usmHMAC384SHA512AuthProtocol) -> do_update_usm_user_info(Key, User#usm_user{auth = Val}); do_update_usm_user_info(_Key, _User, auth, Val) -> {error, {invalid_auth_protocol, Val}}; + do_update_usm_user_info(Key, #usm_user{auth = usmNoAuthProtocol} = User, auth_key, Val) -> @@ -3053,6 +3146,7 @@ do_update_usm_user_info(Key, _ -> {error, {invalid_auth_key, Val}} end; + do_update_usm_user_info(Key, #usm_user{auth = usmHMACMD5AuthProtocol} = User, auth_key, Val) @@ -3072,6 +3166,7 @@ do_update_usm_user_info(_Key, #usm_user{auth = usmHMACMD5AuthProtocol}, auth_key, Val) -> {error, {invalid_auth_key, usmHMACMD5AuthProtocol, Val}}; + do_update_usm_user_info(Key, #usm_user{auth = usmHMACSHAAuthProtocol} = User, auth_key, Val) @@ -3091,6 +3186,87 @@ do_update_usm_user_info(_Key, #usm_user{auth = usmHMACSHAAuthProtocol}, auth_key, Val) -> {error, {invalid_auth_key, usmHMACSHAAuthProtocol, Val}}; + +do_update_usm_user_info(Key, + #usm_user{auth = usmHMAC128SHA224AuthProtocol} = User, + auth_key, Val) + when (length(Val) =:= ?usmHMAC128SHA224AuthProtocol_secret_key_length) -> + case is_crypto_supported(sha224) of + true -> + do_update_usm_user_info(Key, User#usm_user{auth_key = Val}); + false -> + {error, {unsupported_crypto, sha224}} + end; +do_update_usm_user_info(_Key, + #usm_user{auth = usmHMAC128SHA224AuthProtocol = Auth}, + auth_key, Val) when is_list(Val) -> + Len = length(Val), + {error, {invalid_auth_key_length, Auth, Len}}; +do_update_usm_user_info(_Key, + #usm_user{auth = usmHMAC128SHA224AuthProtocol = Auth}, + auth_key, Val) -> + {error, {invalid_auth_key, Auth, Val}}; + +do_update_usm_user_info(Key, + #usm_user{auth = usmHMAC192SHA256AuthProtocol} = User, + auth_key, Val) + when (length(Val) =:= ?usmHMAC192SHA256AuthProtocol_secret_key_length) -> + case is_crypto_supported(sha256) of + true -> + do_update_usm_user_info(Key, User#usm_user{auth_key = Val}); + false -> + {error, {unsupported_crypto, sha256}} + end; +do_update_usm_user_info(_Key, + #usm_user{auth = usmHMAC192SHA256AuthProtocol = Auth}, + auth_key, Val) when is_list(Val) -> + Len = length(Val), + {error, {invalid_auth_key_length, Auth, Len}}; +do_update_usm_user_info(_Key, + #usm_user{auth = usmHMAC192SHA256AuthProtocol = Auth}, + auth_key, Val) -> + {error, {invalid_auth_key, Auth, Val}}; + +do_update_usm_user_info(Key, + #usm_user{auth = usmHMAC256SHA384AuthProtocol} = User, + auth_key, Val) + when (length(Val) =:= ?usmHMAC256SHA384AuthProtocol_secret_key_length) -> + case is_crypto_supported(sha384) of + true -> + do_update_usm_user_info(Key, User#usm_user{auth_key = Val}); + false -> + {error, {unsupported_crypto, sha384}} + end; +do_update_usm_user_info(_Key, + #usm_user{auth = usmHMAC256SHA384AuthProtocol = Auth}, + auth_key, Val) when is_list(Val) -> + Len = length(Val), + {error, {invalid_auth_key_length, Auth, Len}}; +do_update_usm_user_info(_Key, + #usm_user{auth = usmHMAC256SHA384AuthProtocol = Auth}, + auth_key, Val) -> + {error, {invalid_auth_key, Auth, Val}}; + +do_update_usm_user_info(Key, + #usm_user{auth = usmHMAC384SHA512AuthProtocol} = User, + auth_key, Val) + when (length(Val) =:= ?usmHMAC384SHA512AuthProtocol_secret_key_length) -> + case is_crypto_supported(sha512) of + true -> + do_update_usm_user_info(Key, User#usm_user{auth_key = Val}); + false -> + {error, {unsupported_crypto, sha512}} + end; +do_update_usm_user_info(_Key, + #usm_user{auth = usmHMAC384SHA512AuthProtocol = Auth}, + auth_key, Val) when is_list(Val) -> + Len = length(Val), + {error, {invalid_auth_key_length, Auth, Len}}; +do_update_usm_user_info(_Key, + #usm_user{auth = usmHMAC384SHA512AuthProtocol = Auth}, + auth_key, Val) -> + {error, {invalid_auth_key, Auth, Val}}; + do_update_usm_user_info(Key, User, priv, Val) when (Val =:= usmNoPrivProtocol) orelse (Val =:= usmDESPrivProtocol) orelse diff --git a/lib/snmp/src/manager/snmpm_mpd.erl b/lib/snmp/src/manager/snmpm_mpd.erl index a33ae9d710..85821ccb94 100644 --- a/lib/snmp/src/manager/snmpm_mpd.erl +++ b/lib/snmp/src/manager/snmpm_mpd.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2004-2019. All Rights Reserved. +%% Copyright Ericsson AB 2004-2021. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -310,7 +310,16 @@ process_v3_msg(NoteStore, Msg, Hdr, Data, Address, Log) -> {ok, 'version-3', PDU, PduMMS, ok}; _ when is_tuple(Note) -> ?vlog("process_v3_msg -> 7.2.11b: error" - "~n Note: ~p", [Note]), + "~n Note: ~p" + "~n SecEngineID: ~p" + "~n MsgSecModel: ~p" + "~n SecName: ~p" + "~n SecLevel: ~p" + "~n CtxEngineID: ~p" + "~n CtxName: ~p", + [Note, + SecEngineID, MsgSecModel, SecName, SecLevel, + CtxEngineID, CtxName]), Recv = {SecEngineID, MsgSecModel, SecName, SecLevel, CtxEngineID, CtxName, PDU#pdu.request_id}, Err = sec_error(Note, Recv), diff --git a/lib/snmp/src/manager/snmpm_net_if.erl b/lib/snmp/src/manager/snmpm_net_if.erl index 5ff8f91a6e..af78308147 100644 --- a/lib/snmp/src/manager/snmpm_net_if.erl +++ b/lib/snmp/src/manager/snmpm_net_if.erl @@ -303,10 +303,17 @@ do_init(Server, NoteStore) -> %% The 'inet-backend' option has to be first, %% so we might as well add it last. Socket = socket_open(IpPort, InetBackend ++ SocketOpts), + PortInfoStr = + case inet:port(Socket) of + {ok, PortNo} when (IpPort =/= PortNo) -> + lists:flatten(io_lib:format("~p (~w)", [IpPort, PortNo])); + _ -> + lists:flatten(io_lib:format("~p", [IpPort])) + end, ?vtrace("socket created: " - "~n Ip Port: ~p" + "~n Ip Port: ~s" "~n Socket Opts: ~p" - "~n Socket: ~p", [IpPort, SocketOpts, Socket]), + "~n Socket: ~p", [PortInfoStr, SocketOpts, Socket]), #transport{socket = Socket, port_info = IpPort, opts = SocketOpts, @@ -340,7 +347,10 @@ socket_open(IpPort, SocketOpts) -> "~n IpPort: ~p" "~n SocketOpts: ~p", [IpPort, SocketOpts]), case gen_udp:open(IpPort, SocketOpts) of - {error, _} = Error -> + {error, _Reason} = Error -> + ?vlog("socket_open -> entry with" + "~n Options: ~p" + "~n Reason: ~p", [SocketOpts, _Reason]), throw(Error); {ok, Socket} -> Socket diff --git a/lib/snmp/src/manager/snmpm_usm.erl b/lib/snmp/src/manager/snmpm_usm.erl index 6b13c67bea..441618ec86 100644 --- a/lib/snmp/src/manager/snmpm_usm.erl +++ b/lib/snmp/src/manager/snmpm_usm.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2004-2016. All Rights Reserved. +%% Copyright Ericsson AB 2004-2021. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -80,8 +80,8 @@ process_incoming_msg(Packet, Data, SecParams, SecLevel) -> #usmSecurityParameters{msgAuthoritativeEngineID = MsgAuthEngineID, msgUserName = MsgUserName} = UsmSecParams, ?vlog("process_incoming_msg -> [3.2.2]" - "~n authEngineID: ~p" - "~n userName: ~p", [MsgAuthEngineID, MsgUserName]), + "~n authEngineID: ~p" + "~n userName: ~p", [MsgAuthEngineID, MsgUserName]), %% 3.2.3 (b) ?vtrace("process_incoming_msg -> [3.2.3-b] check engine id",[]), @@ -132,14 +132,21 @@ process_incoming_msg(Packet, Data, SecParams, SecLevel) -> authenticate_incoming(Packet, UsmSecParams, UsmUser, SecLevel) -> %% 3.2.6 - ?vtrace("authenticate incoming: 3.2.6",[]), + ?vtrace("authenticate incoming -> 3.2.6" + "~n SecLevel: ~p", [SecLevel]), #usmSecurityParameters{msgAuthoritativeEngineID = MsgAuthEngineID, msgAuthoritativeEngineBoots = MsgAuthEngineBoots, msgAuthoritativeEngineTime = MsgAuthEngineTime, msgAuthenticationParameters = MsgAuthParams} = UsmSecParams, + ?vtrace("authenticate_incoming -> Sec params: " + "~n MsgAuthEngineID: ~w" + "~n MsgAuthEngineBoots: ~p" + "~n MsgAuthEngineTime: ~p", + [MsgAuthEngineID, MsgAuthEngineBoots, MsgAuthEngineTime]), case snmp_misc:is_auth(SecLevel) of true -> + ?vtrace("authenticate_incoming -> authenticate"), SecName = UsmUser#usm_user.sec_name, case is_auth(UsmUser#usm_user.auth, UsmUser#usm_user.auth_key, @@ -156,6 +163,7 @@ authenticate_incoming(Packet, UsmSecParams, UsmUser, SecLevel) -> ?usmStatsWrongDigests_instance, SecName) end; false -> % noAuth + ?vtrace("authenticate_incoming -> don't authenticate"), ok end. @@ -198,13 +206,13 @@ is_auth(AuthProtocol, AuthKey, AuthParams, Packet, SecName, [{securityLevel, 1}]) % authNoPriv end; _ -> %% 3.2.7b - we're non-authoritative - ?vtrace("we are non-authoritative: 3.2.7b",[]), + ?vtrace("is_auth -> we are non-authoritative: 3.2.7b"), SnmpEngineBoots = get_engine_boots(MsgAuthEngineID), - ?vtrace("SnmpEngineBoots: ~p",[SnmpEngineBoots]), + ?vtrace("is_auth -> SnmpEngineBoots: ~p", [SnmpEngineBoots]), SnmpEngineTime = get_engine_time(MsgAuthEngineID), - ?vtrace("SnmpEngineTime: ~p",[SnmpEngineTime]), + ?vtrace("is_auth -> SnmpEngineTime: ~p", [SnmpEngineTime]), LatestRecvTime = get_engine_latest_time(MsgAuthEngineID), - ?vtrace("LatestRecvTime: ~p",[LatestRecvTime]), + ?vtrace("is_auth -> LatestRecvTime: ~p", [LatestRecvTime]), UpdateLCD = if MsgAuthEngineBoots > SnmpEngineBoots -> true; @@ -212,10 +220,11 @@ is_auth(AuthProtocol, AuthKey, AuthParams, Packet, SecName, MsgAuthEngineTime > LatestRecvTime -> true; true -> false end, + ?vtrace("is_auth -> UpdateLCD: ~p", [UpdateLCD]), case UpdateLCD of true -> %% 3.2.7b1 - ?vtrace("update msgAuthoritativeEngineID: 3.2.7b1", - []), + ?vtrace("is_auth -> " + "[3.2.7b1] update msgAuthoritativeEngineID"), set_engine_boots(MsgAuthEngineID, MsgAuthEngineBoots), set_engine_time(MsgAuthEngineID, @@ -226,8 +235,7 @@ is_auth(AuthProtocol, AuthKey, AuthParams, Packet, SecName, ok end, %% 3.2.7.b2 - ?vtrace("check if message is outside time window: 3.2.7b2", - []), + ?vtrace("is_auth -> [3.2.7b2] is message outside time window"), InTimeWindow = if SnmpEngineBoots == 2147483647 -> @@ -242,6 +250,7 @@ is_auth(AuthProtocol, AuthKey, AuthParams, Packet, SecName, {time, MsgAuthEngineTime}]}; true -> true end, + ?vtrace("is_auth -> InTimeWindow: ~p", [InTimeWindow]), case InTimeWindow of {false, Reason} -> ?vinfo("not in time window[3.2.7b2]: ~p", @@ -307,12 +316,18 @@ try_decrypt(usmAesCfb128Protocol, %%----------------------------------------------------------------- generate_outgoing_msg(Message, SecEngineID, SecName, SecData, SecLevel) -> %% 3.1.1 - ?vtrace("generate_outgoing_msg -> entry (3.1.1)",[]), + ?vtrace("generate_outgoing_msg -> entry (3.1.1) when" + "~n SecEngineID: ~p" + "~n SecName: ~p" + "~n SecData: ~p" + "~n SecLevel: ~p", [SecEngineID, SecName, SecData, SecLevel]), {UserName, AuthProtocol, AuthKey, PrivProtocol, PrivKey} = case SecData of [] -> % 3.1.1b + ?vdebug("generate_outgoing_msg -> " + "[(3.1.1b] not a response - read from LCD"), %% Not a response - read from LCD - case snmpm_config:get_usm_user_from_sec_name(SecEngineID, + case snmpm_config:get_usm_user_from_sec_name(SecEngineID, SecName) of {ok, User} -> {User#usm_user.name, @@ -321,6 +336,7 @@ generate_outgoing_msg(Message, SecEngineID, SecName, SecData, SecLevel) -> User#usm_user.priv, User#usm_user.priv_key}; _ -> + ?vlog("generate_outgoing_msg -> (usm) user not found"), error(unknownSecurityName) end; [MsgUserName] -> @@ -330,24 +346,31 @@ generate_outgoing_msg(Message, SecEngineID, SecName, SecData, SecLevel) -> SecData end, SnmpEngineID = get_engine_id(), - ?vtrace("SnmpEngineID: ~p (3.1.6)",[SnmpEngineID]), + ?vtrace("generate_outgoing_msg -> [3.1.6] SnmpEngineID: ~p", [SnmpEngineID]), %% 3.1.6 {MsgAuthEngineBoots, MsgAuthEngineTime} = case snmp_misc:is_auth(SecLevel) of false when SecData == [] -> % not a response {0, 0}; true when SecEngineID /= SnmpEngineID -> + ?vtrace("generate_outgoing_msg -> " + "~n SecEngineID: ~p" + "~n SnmpEngineID: ~p", + [SecEngineID, SnmpEngineID]), {get_engine_boots(SecEngineID), get_engine_time(SecEngineID)}; _ -> + ?vtrace("generate_outgoing_msg -> use local boots and time"), {get_engine_boots(), get_engine_time()} end, %% 3.1.4 - ?vtrace("generate_outgoing_msg -> (3.1.4)",[]), + ?vtrace("generate_outgoing_msg -> [3.1.4]"), ScopedPduBytes = Message#message.data, - {ScopedPduData, MsgPrivParams} = encrypt(ScopedPduBytes, PrivProtocol, PrivKey, SecLevel, MsgAuthEngineBoots, + {ScopedPduData, MsgPrivParams} = encrypt(ScopedPduBytes, + PrivProtocol, PrivKey, + SecLevel, MsgAuthEngineBoots, MsgAuthEngineTime), %% 3.1.5 - 3.1.7 - ?vtrace("generate_outgoing_msg -> (3.1.5 - 3.1.7)",[]), + ?vtrace("generate_outgoing_msg -> [3.1.5 - 3.1.7]",[]), UsmSecParams = #usmSecurityParameters{msgAuthoritativeEngineID = SecEngineID, msgAuthoritativeEngineBoots = MsgAuthEngineBoots, diff --git a/lib/snmp/src/misc/depend.mk b/lib/snmp/src/misc/depend.mk index b2e0a92997..70c5329ae3 100644 --- a/lib/snmp/src/misc/depend.mk +++ b/lib/snmp/src/misc/depend.mk @@ -2,7 +2,7 @@ # %CopyrightBegin% # -# Copyright Ericsson AB 2004-2016. All Rights Reserved. +# Copyright Ericsson AB 2004-2021. All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. @@ -25,6 +25,7 @@ $(EBIN)/snmp_conf.$(EMULATOR): \ $(EBIN)/snmp_config.$(EMULATOR): \ snmp_config.erl \ + snmp_usm.hrl \ ../../include/snmp_types.hrl $(EBIN)/snmp_log.$(EMULATOR): \ @@ -50,6 +51,7 @@ $(EBIN)/snmp_pdu.$(EMULATOR): \ $(EBIN)/snmp_usm.$(EMULATOR): \ snmp_usm.erl \ + snmp_usm.hrl \ snmp_verbosity.hrl \ ../../include/snmp_types.hrl diff --git a/lib/snmp/src/misc/modules.mk b/lib/snmp/src/misc/modules.mk index ac42128724..6534349cd6 100644 --- a/lib/snmp/src/misc/modules.mk +++ b/lib/snmp/src/misc/modules.mk @@ -2,7 +2,7 @@ # %CopyrightBegin% # -# Copyright Ericsson AB 2004-2016. All Rights Reserved. +# Copyright Ericsson AB 2004-2021. All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. @@ -29,6 +29,9 @@ MODULES = \ snmp_usm \ snmp_verbosity -HRLS = snmp_verbosity snmp_debug +HRLS = \ + snmp_verbosity \ + snmp_debug \ + snmp_usm diff --git a/lib/snmp/src/misc/snmp_config.erl b/lib/snmp/src/misc/snmp_config.erl index 43596f1e74..928b469c06 100644 --- a/lib/snmp/src/misc/snmp_config.erl +++ b/lib/snmp/src/misc/snmp_config.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1996-2020. All Rights Reserved. +%% Copyright Ericsson AB 1996-2021. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -22,13 +22,12 @@ -include_lib("kernel/include/file.hrl"). -include("snmp_types.hrl"). +-include("snmp_usm.hrl"). %% Avoid warning for local function error/1 clashing with autoimported BIF. -compile({no_auto_import,[error/1]}). -export([config/0]). -%%-export([write_config_file/4, append_config_file/4, read_config_file/4]). - -export([write_config_file/6, append_config_file/6, read_config_file/4]). -export([write_agent_snmp_files/7, write_agent_snmp_files/12, @@ -946,7 +945,8 @@ config_manager_snmp_usm() -> UserName = ask("7b. User name?", mandatory, fun verify_usm_name/1), SecName = ask("7c. Security name?", UserName, fun verify_usm_sec_name/1), - AuthP = ask("7d. Authentication protocol (no/sha/md5)?", "no", + AuthP = ask("7d. Authentication protocol " + "(no/md5/sha/sha224/sha256/sha384/sha512)?", "no", fun verify_usm_auth_protocol/1), AuthKey = ask_auth_key("7e", AuthP), PrivP = ask("7e. Priv protocol (no/des/aes)?", "no", @@ -1027,12 +1027,36 @@ default_dir(Component, DefDir) -> ask_auth_key(_Prefix, usmNoAuthProtocol) -> ""; +ask_auth_key(Prefix, usmHMACMD5AuthProtocol) -> + ask(Prefix ++ " Authentication [md5] key (length 0 or 16)?", "\"\"", + fun verify_usm_auth_md5_key/1); ask_auth_key(Prefix, usmHMACSHAAuthProtocol) -> ask(Prefix ++ " Authentication [sha] key (length 0 or 20)?", "\"\"", fun verify_usm_auth_sha_key/1); -ask_auth_key(Prefix, usmHMACMD5AuthProtocol) -> - ask(Prefix ++ " Authentication [md5] key (length 0 or 16)?", "\"\"", - fun verify_usm_auth_md5_key/1). +ask_auth_key(Prefix, usmHMAC128SHA224AuthProtocol) -> + ask(Prefix ++ + f(" Authentication [~w] key (length 0 or ~w)?", + [sha224, ?usmHMAC128SHA224AuthProtocol_secret_key_length]), + "\"\"", + fun verify_usm_auth_sha224_key/1); +ask_auth_key(Prefix, usmHMAC192SHA256AuthProtocol) -> + ask(Prefix ++ + f(" Authentication [~w] key (length 0 or ~w)?", + [sha256, ?usmHMAC192SHA256AuthProtocol_secret_key_length]), + "\"\"", + fun verify_usm_auth_sha256_key/1); +ask_auth_key(Prefix, usmHMAC256SHA384AuthProtocol) -> + ask(Prefix ++ + f(" Authentication [~w] key (length 0 or ~w)?", + [sha384, ?usmHMAC256SHA384AuthProtocol_secret_key_length]), + "\"\"", + fun verify_usm_auth_sha384_key/1); +ask_auth_key(Prefix, usmHMAC384SHA512AuthProtocol) -> + ask(Prefix ++ + f(" Authentication [~w] key (length 0 or ~w)?", + [sha512, ?usmHMAC384SHA512AuthProtocol_secret_key_length]), + "\"\"", + fun verify_usm_auth_sha512_key/1). ask_priv_key(_Prefix, usmNoPrivProtocol) -> ""; @@ -1545,18 +1569,38 @@ verify_usm_sec_name(Name) -> verify_usm_auth_protocol("no") -> {ok, usmNoAuthProtocol}; -verify_usm_auth_protocol("sha") -> - {ok, usmHMACSHAAuthProtocol}; verify_usm_auth_protocol("md5") -> {ok, usmHMACMD5AuthProtocol}; +verify_usm_auth_protocol("sha") -> + {ok, usmHMACSHAAuthProtocol}; +verify_usm_auth_protocol("sha224") -> + {ok, usmHMAC128SHA224AuthProtocol}; +verify_usm_auth_protocol("sha256") -> + {ok, usmHMAC192SHA256AuthProtocol}; +verify_usm_auth_protocol("sha384") -> + {ok, usmHMAC256SHA384AuthProtocol}; +verify_usm_auth_protocol("sha512") -> + {ok, usmHMAC384SHA512AuthProtocol}; verify_usm_auth_protocol(AuthP) -> {error, "invalid auth protocol: " ++ AuthP}. +verify_usm_auth_md5_key(Key) -> + verify_usm_key("auth md5", Key, 16). + verify_usm_auth_sha_key(Key) -> verify_usm_key("auth sha", Key, 20). -verify_usm_auth_md5_key(Key) -> - verify_usm_key("auth md5", Key, 16). +verify_usm_auth_sha224_key(Key) -> + verify_usm_key("auth sha224", Key, ?usmHMAC128SHA224AuthProtocol_secret_key_length). + +verify_usm_auth_sha256_key(Key) -> + verify_usm_key("auth sha256", Key, ?usmHMAC192SHA256AuthProtocol_secret_key_length). + +verify_usm_auth_sha384_key(Key) -> + verify_usm_key("auth sha384", Key, ?usmHMAC256SHA384AuthProtocol_secret_key_length). + +verify_usm_auth_sha512_key(Key) -> + verify_usm_key("auth sha512", Key, ?usmHMAC384SHA512AuthProtocol_secret_key_length). verify_usm_priv_protocol("no") -> {ok, usmNoPrivProtocol}; @@ -3041,6 +3085,9 @@ ensure_started(App) -> %% ------------------------------------------------------------------------- +f(F, A) -> + lists:flatten(io_lib:format(F, A)). + d(F, A) -> i("DBG: " ++ F, A). diff --git a/lib/snmp/src/misc/snmp_usm.erl b/lib/snmp/src/misc/snmp_usm.erl index e46993aad4..9652e3a3ff 100644 --- a/lib/snmp/src/misc/snmp_usm.erl +++ b/lib/snmp/src/misc/snmp_usm.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2004-2016. All Rights Reserved. +%% Copyright Ericsson AB 2004-2021. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -34,6 +34,8 @@ -include("snmp_types.hrl"). -include("SNMP-USER-BASED-SM-MIB.hrl"). -include("SNMP-USM-AES-MIB.hrl"). +-include("SNMP-USM-HMAC-SHA2-MIB.hrl"). +-include("snmp_usm.hrl"). -define(VMODULE,"USM"). -include("snmp_verbosity.hrl"). @@ -41,7 +43,15 @@ %%----------------------------------------------------------------- --define(twelwe_zeros, [0,0,0,0,0,0,0,0,0,0,0,0]). +-define(zeros12, [0,0,0,0,0,0,0,0,0,0,0,0]). +-define(zeros16, [0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0]). +-define(zeros24, [0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, + 0,0,0,0,0,0,0,0]). +-define(zeros32, [0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, + 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0]). +-define(zeros48, [0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, + 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, + 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0]). -define(i32(Int), (Int bsr 24) band 255, (Int bsr 16) band 255, (Int bsr 8) band 255, Int band 255). @@ -65,6 +75,14 @@ %% The algorithm is described in appendix A.1 2) of %% rfc2274. %%----------------------------------------------------------------- + +-type algorithm() :: md5 | sha | sha224 | sha256 | sha384 | sha512. + +-spec passwd2localized_key(Alg :: algorithm(), + Passwd :: string(), + EngineID :: string()) -> + LocalizedKey :: list(). + passwd2localized_key(Alg, Passwd, EngineID) when length(Passwd) > 0 -> Key = mk_digest(Alg, Passwd), localize_key(Alg, Key, EngineID). @@ -73,43 +91,38 @@ passwd2localized_key(Alg, Passwd, EngineID) when length(Passwd) > 0 -> %%----------------------------------------------------------------- %% Func: localize_key/3 %% Types: Alg = md5 | sha -%% Passwd = string() +%% Key = binary() %% EngineID = string() %% Purpose: Localizes an unlocalized key for EngineID. See rfc2274 %% section 2.6 for a definition of localized keys. %%----------------------------------------------------------------- + +-spec localize_key(Alg :: algorithm(), + Key :: binary(), + EngineID :: string()) -> + LocalizedKey :: list(). + localize_key(Alg, Key, EngineID) -> Str = [Key, EngineID, Key], binary_to_list(crypto:hash(Alg, Str)). -mk_digest(md5, Passwd) -> - mk_md5_digest(Passwd); -mk_digest(sha, Passwd) -> - mk_sha_digest(Passwd). - -mk_md5_digest(Passwd) -> - Ctx = crypto:hash_init(md5), - Ctx2 = md5_loop(0, [], Ctx, Passwd, length(Passwd)), - crypto:hash_final(Ctx2). - -md5_loop(Count, Buf, Ctx, Passwd, PasswdLen) when Count < 1048576 -> - {Buf64, NBuf} = mk_buf64(length(Buf), Buf, Passwd, PasswdLen), - NCtx = crypto:hash_update(Ctx, Buf64), - md5_loop(Count+64, NBuf, NCtx, Passwd, PasswdLen); -md5_loop(_Count, _Buf, Ctx, _Passwd, _PasswdLen) -> - Ctx. - -mk_sha_digest(Passwd) -> - Ctx = crypto:hash_init(sha), - Ctx2 = sha_loop(0, [], Ctx, Passwd, length(Passwd)), +mk_digest(Alg, Passwd) + when (Alg =:= md5) orelse + (Alg =:= sha) orelse + (Alg =:= sha224) orelse + (Alg =:= sha256) orelse + (Alg =:= sha384) orelse + (Alg =:= sha512) -> + Ctx = crypto:hash_init(Alg), + Ctx2 = mk_digest_loop(0, [], Ctx, Passwd, length(Passwd)), crypto:hash_final(Ctx2). -sha_loop(Count, Buf, Ctx, Passwd, PasswdLen) when Count < 1048576 -> +mk_digest_loop(Count, Buf, Ctx, Passwd, PasswdLen) when Count < 1048576 -> {Buf64, NBuf} = mk_buf64(length(Buf), Buf, Passwd, PasswdLen), NCtx = crypto:hash_update(Ctx, Buf64), - sha_loop(Count+64, NBuf, NCtx, Passwd, PasswdLen); -sha_loop(_Count, _Buf, Ctx, _Passwd, _PasswdLen) -> + mk_digest_loop(Count+64, NBuf, NCtx, Passwd, PasswdLen); +mk_digest_loop(_Count, _Buf, Ctx, _Passwd, _PasswdLen) -> Ctx. %% Create a 64 bytes long string, by repeating Passwd as many times @@ -137,20 +150,122 @@ auth_in(?usmHMACMD5AuthProtocol, AuthKey, AuthParams, Packet) -> auth_in(usmHMACSHAAuthProtocol, AuthKey, AuthParams, Packet) -> sha_auth_in(AuthKey, AuthParams, Packet); auth_in(?usmHMACSHAAuthProtocol, AuthKey, AuthParams, Packet) -> - sha_auth_in(AuthKey, AuthParams, Packet). + sha_auth_in(AuthKey, AuthParams, Packet); +auth_in(usmHMAC128SHA224AuthProtocol, AuthKey, AuthParams, Packet) -> + ?vtrace("auth_in(sha224) -> entry"), + sha2_auth_in(AuthKey, AuthParams, Packet, + sha224, + ?usmHMAC128SHA224AuthProtocol_secret_key_length, + ?usmHMAC128SHA224AuthProtocol_mac_length); +auth_in(?usmHMAC128SHA224AuthProtocol, AuthKey, AuthParams, Packet) -> + ?vtrace("auth_in(sha224) -> entry"), + sha2_auth_in(AuthKey, AuthParams, Packet, + sha224, + ?usmHMAC128SHA224AuthProtocol_secret_key_length, + ?usmHMAC128SHA224AuthProtocol_mac_length); +auth_in(usmHMAC192SHA256AuthProtocol, AuthKey, AuthParams, Packet) -> + ?vtrace("auth_in(sha256) -> entry"), + sha2_auth_in(AuthKey, AuthParams, Packet, + sha256, + ?usmHMAC192SHA256AuthProtocol_secret_key_length, + ?usmHMAC192SHA256AuthProtocol_mac_length); +auth_in(?usmHMAC192SHA256AuthProtocol, AuthKey, AuthParams, Packet) -> + ?vtrace("auth_in(sha256) -> entry"), + sha2_auth_in(AuthKey, AuthParams, Packet, + sha256, + ?usmHMAC192SHA256AuthProtocol_secret_key_length, + ?usmHMAC192SHA256AuthProtocol_mac_length); +auth_in(usmHMAC256SHA384AuthProtocol, AuthKey, AuthParams, Packet) -> + ?vtrace("auth_in(sha384) -> entry"), + sha2_auth_in(AuthKey, AuthParams, Packet, + sha384, + ?usmHMAC256SHA384AuthProtocol_secret_key_length, + ?usmHMAC256SHA384AuthProtocol_mac_length); +auth_in(?usmHMAC256SHA384AuthProtocol, AuthKey, AuthParams, Packet) -> + ?vtrace("auth_in(sha384) -> entry"), + sha2_auth_in(AuthKey, AuthParams, Packet, + sha384, + ?usmHMAC256SHA384AuthProtocol_secret_key_length, + ?usmHMAC256SHA384AuthProtocol_mac_length); +auth_in(usmHMAC384SHA512AuthProtocol, AuthKey, AuthParams, Packet) -> + ?vtrace("auth_in(sha512) -> entry"), + sha2_auth_in(AuthKey, AuthParams, Packet, + sha512, + ?usmHMAC384SHA512AuthProtocol_secret_key_length, + ?usmHMAC384SHA512AuthProtocol_mac_length); +auth_in(?usmHMAC384SHA512AuthProtocol, AuthKey, AuthParams, Packet) -> + ?vtrace("auth_in(sha512) -> entry"), + sha2_auth_in(AuthKey, AuthParams, Packet, + sha512, + ?usmHMAC384SHA512AuthProtocol_secret_key_length, + ?usmHMAC384SHA512AuthProtocol_mac_length). auth_out(usmNoAuthProtocol, _AuthKey, _Message, _UsmSecParams) -> % 3.1.3 + %% ?vtrace("auth_out(noAuth) -> entry"), error(unSupportedSecurityLevel); auth_out(?usmNoAuthProtocol, _AuthKey, _Message, _UsmSecParams) -> % 3.1.3 + %% ?vtrace("auth_out(noAuth) -> entry"), error(unSupportedSecurityLevel); auth_out(usmHMACMD5AuthProtocol, AuthKey, Message, UsmSecParams) -> + %% ?vtrace("auth_out(md5) -> entry"), md5_auth_out(AuthKey, Message, UsmSecParams); auth_out(?usmHMACMD5AuthProtocol, AuthKey, Message, UsmSecParams) -> + %% ?vtrace("auth_out(md5) -> entry"), md5_auth_out(AuthKey, Message, UsmSecParams); auth_out(usmHMACSHAAuthProtocol, AuthKey, Message, UsmSecParams) -> + %% ?vtrace("auth_out(sha) -> entry"), sha_auth_out(AuthKey, Message, UsmSecParams); auth_out(?usmHMACSHAAuthProtocol, AuthKey, Message, UsmSecParams) -> - sha_auth_out(AuthKey, Message, UsmSecParams). + %% ?vtrace("auth_out(sha) -> entry"), + sha_auth_out(AuthKey, Message, UsmSecParams); +auth_out(usmHMAC128SHA224AuthProtocol, AuthKey, Message, UsmSecParams) -> + %% ?vtrace("auth_out(sha224) -> entry"), + sha2_auth_out(AuthKey, Message, UsmSecParams, + sha224, + ?usmHMAC128SHA224AuthProtocol_secret_key_length, + ?usmHMAC128SHA224AuthProtocol_mac_length); +auth_out(?usmHMAC128SHA224AuthProtocol, AuthKey, Message, UsmSecParams) -> + %% ?vtrace("auth_out(sha224) -> entry"), + sha2_auth_out(AuthKey, Message, UsmSecParams, + sha224, + ?usmHMAC128SHA224AuthProtocol_secret_key_length, + ?usmHMAC128SHA224AuthProtocol_mac_length); +auth_out(usmHMAC192SHA256AuthProtocol, AuthKey, Message, UsmSecParams) -> + %% ?vtrace("auth_out(sha256) -> entry"), + sha2_auth_out(AuthKey, Message, UsmSecParams, + sha256, + ?usmHMAC192SHA256AuthProtocol_secret_key_length, + ?usmHMAC192SHA256AuthProtocol_mac_length); +auth_out(?usmHMAC192SHA256AuthProtocol, AuthKey, Message, UsmSecParams) -> + %% ?vtrace("auth_out(sha256) -> entry"), + sha2_auth_out(AuthKey, Message, UsmSecParams, + sha256, + ?usmHMAC192SHA256AuthProtocol_secret_key_length, + ?usmHMAC192SHA256AuthProtocol_mac_length); +auth_out(usmHMAC256SHA384AuthProtocol, AuthKey, Message, UsmSecParams) -> + %% ?vtrace("auth_out(sha384) -> entry"), + sha2_auth_out(AuthKey, Message, UsmSecParams, + sha384, + ?usmHMAC256SHA384AuthProtocol_secret_key_length, + ?usmHMAC256SHA384AuthProtocol_mac_length); +auth_out(?usmHMAC256SHA384AuthProtocol, AuthKey, Message, UsmSecParams) -> + %% ?vtrace("auth_out(sha384) -> entry"), + sha2_auth_out(AuthKey, Message, UsmSecParams, + sha384, + ?usmHMAC256SHA384AuthProtocol_secret_key_length, + ?usmHMAC256SHA384AuthProtocol_mac_length); +auth_out(usmHMAC384SHA512AuthProtocol, AuthKey, Message, UsmSecParams) -> + %% ?vtrace("auth_out(sha512) -> entry"), + sha2_auth_out(AuthKey, Message, UsmSecParams, + sha512, + ?usmHMAC384SHA512AuthProtocol_secret_key_length, + ?usmHMAC384SHA512AuthProtocol_mac_length); +auth_out(?usmHMAC384SHA512AuthProtocol, AuthKey, Message, UsmSecParams) -> + %% ?vtrace("auth_out(sha512) -> entry"), + sha2_auth_out(AuthKey, Message, UsmSecParams, + sha512, + ?usmHMAC384SHA512AuthProtocol_secret_key_length, + ?usmHMAC384SHA512AuthProtocol_mac_length). md5_auth_out(AuthKey, Message, UsmSecParams) -> %% ?vtrace("md5_auth_out -> entry with" @@ -158,7 +273,7 @@ md5_auth_out(AuthKey, Message, UsmSecParams) -> %% "~n Message: ~w" %% "~n UsmSecParams: ~w", [AuthKey, Message, UsmSecParams]), %% 6.3.1.1 - Message2 = set_msg_auth_params(Message, UsmSecParams, ?twelwe_zeros), + Message2 = set_msg_auth_params(Message, UsmSecParams, ?zeros12), Packet = snmp_pdus:enc_message_only(Message2), %% 6.3.1.2-4 is done by the crypto function %% 6.3.1.4 @@ -174,7 +289,7 @@ md5_auth_in(AuthKey, AuthParams, Packet) when length(AuthParams) == 12 -> %% "~n AuthParams: ~w" %% "~n Packet: ~w", [AuthKey, AuthParams, Packet]), %% 6.3.2.3 - Packet2 = patch_packet(binary_to_list(Packet)), + Packet2 = patch_packet(binary_to_list(Packet), ?zeros12), %% 6.3.2.5 MAC = binary_to_list(crypto:macN(hmac, md5, AuthKey, Packet2, 12)), %% 6.3.2.6 @@ -183,15 +298,15 @@ md5_auth_in(AuthKey, AuthParams, Packet) when length(AuthParams) == 12 -> MAC == AuthParams; md5_auth_in(_AuthKey, _AuthParams, _Packet) -> %% 6.3.2.1 - ?vtrace("md5_auth_in -> entry with" - "~n _AuthKey: ~p" - "~n _AuthParams: ~p", [_AuthKey, _AuthParams]), + ?vlog("md5_auth_in -> entry with" + "~n _AuthKey: ~p" + "~n _AuthParams: ~p", [_AuthKey, _AuthParams]), false. sha_auth_out(AuthKey, Message, UsmSecParams) -> %% 7.3.1.1 - Message2 = set_msg_auth_params(Message, UsmSecParams, ?twelwe_zeros), + Message2 = set_msg_auth_params(Message, UsmSecParams, ?zeros12), Packet = snmp_pdus:enc_message_only(Message2), %% 7.3.1.2-4 is done by the crypto function %% 7.3.1.4 @@ -201,19 +316,80 @@ sha_auth_out(AuthKey, Message, UsmSecParams) -> sha_auth_in(AuthKey, AuthParams, Packet) when length(AuthParams) =:= 12 -> %% 7.3.2.3 - Packet2 = patch_packet(binary_to_list(Packet)), + Packet2 = patch_packet(binary_to_list(Packet), ?zeros12), %% 7.3.2.5 MAC = binary_to_list(crypto:macN(hmac, sha, AuthKey, Packet2, 12)), %% 7.3.2.6 MAC == AuthParams; sha_auth_in(_AuthKey, _AuthParams, _Packet) -> %% 7.3.2.1 - ?vtrace("sha_auth_in -> entry with" - "~n _AuthKey: ~p" - "~n _AuthParams: ~p", [_AuthKey, _AuthParams]), + ?vlog("sha_auth_in -> entry with" + "~n _AuthKey: ~p" + "~n _AuthParams: ~p", [_AuthKey, _AuthParams]), + false. + + +%% RFC:4.2.1 +sha2_auth_out(AuthKey, Message, UsmSecParams, + HashAlg, M, N) + when (length(AuthKey) =:= M) -> + %% ?vtrace("sha2_auth_out -> entry with" + %% "~n AuthKey: ~p" + %% "~n UsmSecParams: ~p" + %% "~n HashAlg: ~p" + %% "~n M: ~p" + %% "~n N: ~p", + %% [AuthKey, UsmSecParams, HashAlg, M, N]), + %% 4.2.1:1 + Message2 = set_msg_auth_params(Message, UsmSecParams, zeros(N)), + Packet = snmp_pdus:enc_message_only(Message2), + %% 7.3.1.2-4 is done by the crypto function + %% 4.2.1:2-3 + MAC = binary_to_list(crypto:macN(hmac, HashAlg, AuthKey, Packet, N)), + %% ?vtrace("sha2_auth_out -> " + %% "~n MAC: ~p", [MAC]), + %% 4.2.1:4-5 + set_msg_auth_params(Message, UsmSecParams, MAC). + +%% RFC:4.2.2 +sha2_auth_in(AuthKey, AuthParams, Packet, + HashAlg, M, N) + when (length(AuthKey) =:= M) andalso + (length(AuthParams) =:= N) -> + %% ?vtrace("sha2_auth_in -> entry with" + %% "~n AuthKey: ~p" + %% "~n AuthParams: ~p" + %% "~n HashAlg: ~p" + %% "~n M: ~p" + %% "~n N: ~p", [AuthKey, AuthParams, HashAlg, M, N]), + %% 4.2.2:3 + Packet2 = patch_packet(binary_to_list(Packet), zeros(N)), + %% 4.2.2:4 + MAC = binary_to_list(crypto:macN(hmac, HashAlg, AuthKey, Packet2, N)), + %% ?vtrace("sha2_auth_in -> " + %% "~n MAC: ~p", [MAC]), + %% 4.2.2:7 + %% true | {false, authenticationFailure} + (MAC == AuthParams); +sha2_auth_in(_AuthKey, _AuthParams, _Packet, + _HashAlg, _M, _N) -> + %% 4.2.2 + ?vlog("sha2_auth_in -> entry with" + "~n _AuthKey: ~p" + "~n _AuthParams: ~p" + "~n _HashAlg: ~p" + "~n _M: ~p" + "~n _N: ~p", [_AuthKey, _AuthParams, _HashAlg, _M, _N]), + %% authenticationError. false. +zeros(16) -> ?zeros16; +zeros(24) -> ?zeros24; +zeros(32) -> ?zeros32; +zeros(48) -> ?zeros48. + + des_encrypt(PrivKey, Data, SaltFun) -> [A,B,C,D,E,F,G,H | PreIV] = PrivKey, DesKey = [A,B,C,D,E,F,G,H], @@ -289,58 +465,61 @@ set_msg_auth_params(Message, UsmSecParams, AuthParams) -> %% Not very nice... %% This function patches the asn.1 encoded message. It changes the -%% AuthenticationParameters to 12 zeros. +%% AuthenticationParameters to 12|16|24|32|48 zeros. %% NOTE: returns a deep list of bytes -patch_packet([48 | T]) -> +patch_packet([48 | T], Patch) -> %% Length for whole packet - 2 is tag for version {Len1, [2 | T1]} = split_len(T), %% Length for version - 48 is tag for header data {Len2, [Vsn,48|T2]} = split_len(T1), %% Length for header data {Len3, T3} = split_len(T2), - [48,Len1,2,Len2,Vsn,48,Len3|pp2(dec_len(Len3),T3)]. + [48,Len1,2,Len2,Vsn,48,Len3|pp2(dec_len(Len3),T3,Patch)]. %% Skip HeaderData - 4 is tag for SecurityParameters -pp2(0,[4|T]) -> +pp2(0,[4|T],Patch) -> %% 48 is tag for UsmSecParams {Len1,[48|T1]} = split_len(T), %% 4 is tag for EngineID {Len2,[4|T2]} = split_len(T1), %% Len 3 is length for EngineID {Len3,T3} = split_len(T2), - [4,Len1,48,Len2,4,Len3|pp3(dec_len(Len3),T3)]; -pp2(N,[H|T]) -> - [H|pp2(N-1,T)]. + [4,Len1,48,Len2,4,Len3|pp3(dec_len(Len3),T3,Patch)]; +pp2(N,[H|T],Patch) -> + [H|pp2(N-1,T,Patch)]. %% Skip EngineID - 2 is tag for EngineBoots -pp3(0,[2|T]) -> +pp3(0,[2|T],Patch) -> {Len1,T1} = split_len(T), - [2,Len1|pp4(dec_len(Len1),T1)]; -pp3(N,[H|T]) -> - [H|pp3(N-1,T)]. + [2,Len1|pp4(dec_len(Len1),T1,Patch)]; +pp3(N,[H|T],Patch) -> + [H|pp3(N-1,T,Patch)]. %% Skip EngineBoots - 2 is tag for EngineTime -pp4(0,[2|T]) -> +pp4(0,[2|T],Patch) -> {Len1,T1} = split_len(T), - [2,Len1|pp5(dec_len(Len1),T1)]; -pp4(N,[H|T]) -> - [H|pp4(N-1,T)]. + [2,Len1|pp5(dec_len(Len1),T1,Patch)]; +pp4(N,[H|T],Patch) -> + [H|pp4(N-1,T,Patch)]. %% Skip EngineTime - 4 is tag for UserName -pp5(0,[4|T]) -> +pp5(0,[4|T],Patch) -> {Len1,T1} = split_len(T), - [4,Len1|pp6(dec_len(Len1),T1)]; -pp5(N,[H|T]) -> - [H|pp5(N-1,T)]. + [4,Len1|pp6(dec_len(Len1),T1,Patch)]; +pp5(N,[H|T],Patch) -> + [H|pp5(N-1,T,Patch)]. %% Skip UserName - 4 is tag for AuthenticationParameters %% This is what we're looking for! -pp6(0,[4|T]) -> - {Len1,[_,_,_,_,_,_,_,_,_,_,_,_|T1]} = split_len(T), - 12 = dec_len(Len1), - [4,Len1,?twelwe_zeros|T1]; -pp6(N,[H|T]) -> - [H|pp6(N-1,T)]. +pp6(0, [4|T], Patch) -> + PatchLen = length(Patch), + {Len1, Rest1} = split_len(T), + PatchLen = dec_len(Len1), % Force match + {_Old, Rest2} = lists:split(PatchLen, Rest1), + %% Patch: list of zeros of length: 12 | 16 | 24 | 32 | 48 + [4, Len1, Patch|Rest2]; +pp6(N,[H|T],Patch) -> + [H|pp6(N-1,T,Patch)]. %% Returns {LengthOctets, Rest} diff --git a/lib/snmp/src/misc/snmp_usm.hrl b/lib/snmp/src/misc/snmp_usm.hrl new file mode 100644 index 0000000000..8e537b1494 --- /dev/null +++ b/lib/snmp/src/misc/snmp_usm.hrl @@ -0,0 +1,52 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 2021-2021. All Rights Reserved. +%% +%% Licensed under the Apache License, Version 2.0 (the "License"); +%% you may not use this file except in compliance with the License. +%% You may obtain a copy of the License at +%% +%% http://www.apache.org/licenses/LICENSE-2.0 +%% +%% Unless required by applicable law or agreed to in writing, software +%% distributed under the License is distributed on an "AS IS" BASIS, +%% WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +%% See the License for the specific language governing permissions and +%% limitations under the License. +%% +%% %CopyrightEnd% +%% + +-ifndef(snmp_usm). + +%% RFC7860:4.2 +%% Values of constants M (the length of the secret key in octets) and N +%% (the length of the Message Authentication Code (MAC) output in +%% octets), and the hash function H used below are: +%% +%% usmHMAC128SHA224AuthProtocol: M=28, N=16, H=SHA-224; +%% +%% usmHMAC192SHA256AuthProtocol: M=32, N=24, H=SHA-256; +%% +%% usmHMAC256SHA384AuthProtocol: M=48, N=32, H=SHA-384; +%% +%% usmHMAC384SHA512AuthProtocol: M=64, N=48, H=SHA-512. +%% +-define(usmHMAC128SHA224AuthProtocol_secret_key_length, 28). +-define(usmHMAC128SHA224AuthProtocol_mac_length, 16). + +-define(usmHMAC192SHA256AuthProtocol_secret_key_length, 32). +-define(usmHMAC192SHA256AuthProtocol_mac_length, 24). + +-define(usmHMAC256SHA384AuthProtocol_secret_key_length, 48). +-define(usmHMAC256SHA384AuthProtocol_mac_length, 32). + +-define(usmHMAC384SHA512AuthProtocol_secret_key_length, 64). +-define(usmHMAC384SHA512AuthProtocol_mac_length, 48). + +-endif. % -ifndef(snmp_usm). + + + + diff --git a/lib/snmp/test/snmp_agent_SUITE.erl b/lib/snmp/test/snmp_agent_SUITE.erl index 6564c48d70..39d7a754a0 100644 --- a/lib/snmp/test/snmp_agent_SUITE.erl +++ b/lib/snmp/test/snmp_agent_SUITE.erl @@ -202,6 +202,10 @@ v3_crypto_basic/1, v3_md5_auth/1, v3_sha_auth/1, + v3_sha224_auth/1, + v3_sha256_auth/1, + v3_sha384_auth/1, + v3_sha512_auth/1, v3_des_priv/1, %% major_tcs - test_multi_threaded, test_multi_threaded_ext @@ -3368,6 +3372,10 @@ v3_security_cases() -> v3_crypto_basic, v3_md5_auth, v3_sha_auth, + v3_sha224_auth, + v3_sha256_auth, + v3_sha384_auth, + v3_sha512_auth, v3_des_priv ]. @@ -3461,6 +3469,8 @@ v3_sha_auth(Config) when is_list(Config) -> MA = whereis(snmp_master_agent), + snmp_user_based_sm_mib:usmUserTable(print), + ?line load_master("Test2"), ?line load_master("TestTrap"), ?line load_master("TestTrapv2"), @@ -3475,6 +3485,122 @@ v3_sha_auth(Config) when is_list(Config) -> ?line unload_master("Test2"), ?line reset_target_params_conf(AgentConfDir). +v3_sha224_auth(suite) -> []; +v3_sha224_auth(Config) when is_list(Config) -> + ?P(v3_sha224_auth), + init_case(Config), + + ?NPRINT("Testing SHA224 authentication...takes a few seconds..."), + + AgentConfDir = ?config(agent_conf_dir, Config), + ?line rewrite_target_params_conf(AgentConfDir, "authSHA224", authNoPriv), + ?line snmp_target_mib:reconfigure(AgentConfDir), + + MA = whereis(snmp_master_agent), + + snmp_user_based_sm_mib:usmUserTable(print), + + ?line load_master("Test2"), + ?line load_master("TestTrap"), + ?line load_master("TestTrapv2"), + + try_test(v3_sync, [[{v2_proc, []}, + {ma_v2_trap1, [MA]}, + {v3_inform_sync, [MA]}]], + [{sec_level, authNoPriv}, {user, "authSHA224"}]), + + ?line unload_master("TestTrapv2"), + ?line unload_master("TestTrap"), + ?line unload_master("Test2"), + ?line reset_target_params_conf(AgentConfDir). + +v3_sha256_auth(suite) -> []; +v3_sha256_auth(Config) when is_list(Config) -> + ?P(v3_sha256_auth), + init_case(Config), + + ?NPRINT("Testing SHA256 authentication...takes a few seconds..."), + + AgentConfDir = ?config(agent_conf_dir, Config), + ?line rewrite_target_params_conf(AgentConfDir, "authSHA256", authNoPriv), + ?line snmp_target_mib:reconfigure(AgentConfDir), + + MA = whereis(snmp_master_agent), + + snmp_user_based_sm_mib:usmUserTable(print), + + ?line load_master("Test2"), + ?line load_master("TestTrap"), + ?line load_master("TestTrapv2"), + + try_test(v3_sync, [[{v2_proc, []}, + {ma_v2_trap1, [MA]}, + {v3_inform_sync, [MA]}]], + [{sec_level, authNoPriv}, {user, "authSHA256"}]), + + ?line unload_master("TestTrapv2"), + ?line unload_master("TestTrap"), + ?line unload_master("Test2"), + ?line reset_target_params_conf(AgentConfDir). + +v3_sha384_auth(suite) -> []; +v3_sha384_auth(Config) when is_list(Config) -> + ?P(v3_sha384_auth), + init_case(Config), + + ?NPRINT("Testing SHA authentication...takes a few seconds..."), + + AgentConfDir = ?config(agent_conf_dir, Config), + ?line rewrite_target_params_conf(AgentConfDir, "authSHA384", authNoPriv), + ?line snmp_target_mib:reconfigure(AgentConfDir), + + MA = whereis(snmp_master_agent), + + snmp_user_based_sm_mib:usmUserTable(print), + + ?line load_master("Test2"), + ?line load_master("TestTrap"), + ?line load_master("TestTrapv2"), + + try_test(v3_sync, [[{v2_proc, []}, + {ma_v2_trap1, [MA]}, + {v3_inform_sync, [MA]}]], + [{sec_level, authNoPriv}, {user, "authSHA384"}]), + + ?line unload_master("TestTrapv2"), + ?line unload_master("TestTrap"), + ?line unload_master("Test2"), + ?line reset_target_params_conf(AgentConfDir). + +v3_sha512_auth(suite) -> []; +v3_sha512_auth(Config) when is_list(Config) -> + ?P(v3_sha512_auth), + init_case(Config), + + ?NPRINT("Testing SHA512 authentication...takes a few seconds..."), + + AgentConfDir = ?config(agent_conf_dir, Config), + ?line rewrite_target_params_conf(AgentConfDir, "authSHA512", authNoPriv), + ?line snmp_target_mib:reconfigure(AgentConfDir), + + MA = whereis(snmp_master_agent), + + snmp_user_based_sm_mib:usmUserTable(print), + + ?line load_master("Test2"), + ?line load_master("TestTrap"), + ?line load_master("TestTrapv2"), + + try_test(v3_sync, [[{v2_proc, []}, + {ma_v2_trap1, [MA]}, + {v3_inform_sync, [MA]}]], + [{sec_level, authNoPriv}, {user, "authSHA512"}]), + + ?line unload_master("TestTrapv2"), + ?line unload_master("TestTrap"), + ?line unload_master("Test2"), + ?line reset_target_params_conf(AgentConfDir). + v3_des_priv(suite) -> []; v3_des_priv(Config) when is_list(Config) -> ?P(v3_des_priv), diff --git a/lib/snmp/test/snmp_agent_mibs_SUITE.erl b/lib/snmp/test/snmp_agent_mibs_SUITE.erl index ce6ec80322..587779db94 100644 --- a/lib/snmp/test/snmp_agent_mibs_SUITE.erl +++ b/lib/snmp/test/snmp_agent_mibs_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2003-2020. All Rights Reserved. +%% Copyright Ericsson AB 2003-2021. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -380,13 +380,13 @@ do_size_check(Config) -> Verbosity = trace, MibStorage = ?config(mib_storage, Config), - ?DBG("do_size_check -> MibStorage: ~p", [MibStorage]), + ?IPRINT("do_size_check -> MibStorage: ~p", [MibStorage]), MibDir = ?config(data_dir, Config), StdMibDir = filename:join(code:priv_dir(snmp), "mibs") ++ "/", - ?DBG("do_size_check -> start symbolic store", []), + ?IPRINT("do_size_check -> start symbolic store", []), ?line sym_start(Prio, MibStorage, Verbosity), - ?DBG("do_size_check -> start mib server", []), + ?IPRINT("do_size_check -> start mib server", []), ?line MibsPid = mibs_start(Prio, MibStorage, Verbosity), Mibs = ["Test2", "TestTrap", "TestTrapv2"], @@ -402,23 +402,23 @@ do_size_check(Config) -> "SNMPv2-TC", "SNMPv2-TM"], - ?DBG("do_size_check -> load mibs", []), - ?line load_mibs(MibsPid, MibDir, Mibs), - ?DBG("do_size_check -> load std mibs", []), + ?IPRINT("do_size_check -> load std mibs", []), ?line load_mibs(MibsPid, StdMibDir, StdMibs), + ?IPRINT("do_size_check -> load (own) mibs", []), + ?line load_mibs(MibsPid, MibDir, Mibs), ?SLEEP(2000), - ?DBG("do_size_check -> display mem usage", []), + ?IPRINT("do_size_check -> display mem usage", []), ?line display_memory_usage(MibsPid), - ?DBG("do_size_check -> unload std mibs", []), - ?line unload_mibs(MibsPid, StdMibs), - ?DBG("do_size_check -> unload mibs", []), + ?IPRINT("do_size_check -> unload (own) mibs", []), ?line unload_mibs(MibsPid, Mibs), + ?IPRINT("do_size_check -> unload std mibs", []), + ?line unload_mibs(MibsPid, StdMibs), - ?DBG("do_size_check -> stop mib server", []), + ?IPRINT("do_size_check -> stop mib server", []), ?line mibs_stop(MibsPid), - ?DBG("do_size_check -> stop symbolic store", []), + ?IPRINT("do_size_check -> stop symbolic store", []), ?line sym_stop(), ?IPRINT("do_size_check -> done", []), @@ -1102,27 +1102,44 @@ tc_try(Name, Init, TC) monitor_node(Node, true), Pid = spawn_link(Node, TC), receive + %% No test case could/would provoke a nodedown => SKIP {nodedown, Node} = N -> - exit(N); + ?NPRINT("unexpected node down ~p", [Node]), + exit({skip, N}); {'EXIT', Pid, normal} -> monitor_node(Node, false), ok; {'EXIT', Pid, ok} -> monitor_node(Node, false), ok; + %% This is a node down. + %% Its just a race that this came before the actual + %% 'nodedown'. + %% Also, there is no normal test case that could provoke + %% a node down, so this is *not* our fault => SKIP + {'EXIT', Pid, noconnection = Reason} -> + ?NPRINT("unexpected '~p' termination", [Reason]), + monitor_node(Node, false), % Just in case + exit({skip, Reason}); {'EXIT', Pid, Reason} -> - monitor_node(Node, false), + monitor_node(Node, false), exit(Reason) end end, Post = fun(Node) -> - monitor_node(Node, true), - ?NPRINT("try stop node ~p", [Node]), - ?STOP_NODE(Node), receive {nodedown, Node} -> - ?NPRINT("node ~p stopped", [Node]), + ?NPRINT("node ~p (already) stopped", [Node]), ok + after 0 -> + monitor_node(Node, true), + ?NPRINT("try stop node ~p", [Node]), + ?STOP_NODE(Node), + receive + {nodedown, Node} -> + ?NPRINT("node ~p stopped", [Node]), + ok + end end end, ?TC_TRY(Name, Pre, Case, Post). diff --git a/lib/snmp/test/snmp_agent_test_lib.erl b/lib/snmp/test/snmp_agent_test_lib.erl index 236015cfe0..2f96a90235 100644 --- a/lib/snmp/test/snmp_agent_test_lib.erl +++ b/lib/snmp/test/snmp_agent_test_lib.erl @@ -1750,6 +1750,26 @@ update_usm(Vsns, Dir) -> usmNoPrivProtocol, "", "", "", "passwd_shaxxxxxxxxxx", ""}, + {"agentEngine", "authSHA224", "authSHA224", zeroDotZero, + usmHMAC128SHA224AuthProtocol, "", "", + usmNoPrivProtocol, "", "", "", + "passwd_sha224xxxxxxxxxxxxxxx", ""}, + + {"agentEngine", "authSHA256", "authSHA256", zeroDotZero, + usmHMAC192SHA256AuthProtocol, "", "", + usmNoPrivProtocol, "", "", "", + "passwd_sha256xxxxxxxxxxxxxxxxxxx", ""}, + + {"agentEngine", "authSHA384", "authSHA384", zeroDotZero, + usmHMAC256SHA384AuthProtocol, "", "", + usmNoPrivProtocol, "", "", "", + "passwd_sha384xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx", ""}, + + {"agentEngine", "authSHA512", "authSHA512", zeroDotZero, + usmHMAC384SHA512AuthProtocol, "", "", + usmNoPrivProtocol, "", "", "", + "passwd_sha512xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx", ""}, + {"agentEngine", "privDES", "privDES", zeroDotZero, usmHMACSHAAuthProtocol, "", "", usmDESPrivProtocol, "", "", "", @@ -1772,6 +1792,26 @@ update_usm(Vsns, Dir) -> usmNoPrivProtocol, "", "", "", "passwd_shaxxxxxxxxxx", ""}, + {"mgrEngine", "authSHA224", "authSHA224", zeroDotZero, + usmHMAC128SHA224AuthProtocol, "", "", + usmNoPrivProtocol, "", "", "", + "passwd_sha224xxxxxxxxxxxxxxx", ""}, + + {"mgrEngine", "authSHA256", "authSHA256", zeroDotZero, + usmHMAC192SHA256AuthProtocol, "", "", + usmNoPrivProtocol, "", "", "", + "passwd_sha256xxxxxxxxxxxxxxxxxxx", ""}, + + {"mgrEngine", "authSHA384", "authSHA384", zeroDotZero, + usmHMAC256SHA384AuthProtocol, "", "", + usmNoPrivProtocol, "", "", "", + "passwd_sha384xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx", ""}, + + {"mgrEngine", "authSHA512", "authSHA512", zeroDotZero, + usmHMAC384SHA512AuthProtocol, "", "", + usmNoPrivProtocol, "", "", "", + "passwd_sha512xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx", ""}, + {"mgrEngine", "privDES", "privDES", zeroDotZero, usmHMACSHAAuthProtocol, "", "", usmDESPrivProtocol, "", "", "", @@ -1826,10 +1866,14 @@ update_community(_, Dir) -> -define(tDescr_instance, [1,3,6,1,2,1,16,1,0]). update_vacm(_Vsn, Dir) -> - Conf = [{vacmSecurityToGroup, usm, "authMD5", "initial"}, - {vacmSecurityToGroup, usm, "authSHA", "initial"}, - {vacmSecurityToGroup, usm, "privDES", "initial"}, - {vacmSecurityToGroup, usm, "newUser", "initial"}, + Conf = [{vacmSecurityToGroup, usm, "authMD5", "initial"}, + {vacmSecurityToGroup, usm, "authSHA", "initial"}, + {vacmSecurityToGroup, usm, "authSHA224", "initial"}, + {vacmSecurityToGroup, usm, "authSHA256", "initial"}, + {vacmSecurityToGroup, usm, "authSHA384", "initial"}, + {vacmSecurityToGroup, usm, "authSHA512", "initial"}, + {vacmSecurityToGroup, usm, "privDES", "initial"}, + {vacmSecurityToGroup, usm, "newUser", "initial"}, {vacmViewTreeFamily, "internet", ?tDescr_instance, excluded, null}], ?line ok = snmp_config:update_agent_vacm_config(Dir, Conf). diff --git a/lib/snmp/test/snmp_manager_SUITE.erl b/lib/snmp/test/snmp_manager_SUITE.erl index 15c02942ba..af333f0833 100644 --- a/lib/snmp/test/snmp_manager_SUITE.erl +++ b/lib/snmp/test/snmp_manager_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2003-2020. All Rights Reserved. +%% Copyright Ericsson AB 2003-2021. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -38,6 +38,7 @@ -include_lib("snmp/include/snmp_types.hrl"). -include_lib("snmp/include/STANDARD-MIB.hrl"). +-include_lib("snmp/include/SNMP-USER-BASED-SM-MIB.hrl"). -include_lib("snmp/src/manager/snmpm_internal.hrl"). @@ -67,6 +68,10 @@ info/1, usm_priv_aes/1, + usm_sha224_priv_aes/1, + usm_sha256_priv_aes/1, + usm_sha384_priv_aes/1, + usm_sha512_priv_aes/1, simple_sync_get3/1, simple_async_get3/1, @@ -86,6 +91,13 @@ simple_async_get_bulk3_cbp_temp/1, simple_async_get_bulk3_cbp_perm/1, + simple_v3_exchange_md5/1, + simple_v3_exchange_sha/1, + simple_v3_exchange_sha224/1, + simple_v3_exchange_sha256/1, + simple_v3_exchange_sha384/1, + simple_v3_exchange_sha512/1, + discovery/1, trap1/1, @@ -184,8 +196,9 @@ groups() -> {otp8015, [], otp8015_cases()}, {otp8395, [], otp8395_cases()}, {ipv6, [], ipv6_tests()}, - {ipv6_mt, [], ipv6_tests()} + {ipv6_mt, [], ipv6_tests()}, + {v3, [], v3_cases()} ]. inet_backend_default_cases() -> @@ -210,7 +223,8 @@ all_cases() -> discovery, {group, tickets}, {group, ipv6}, - {group, ipv6_mt} + {group, ipv6_mt}, + {group, v3} ]. start_and_stop_tests_cases() -> @@ -227,7 +241,11 @@ start_and_stop_tests_cases() -> misc_tests_cases() -> [ info, - usm_priv_aes + usm_priv_aes, + usm_sha224_priv_aes, + usm_sha256_priv_aes, + usm_sha384_priv_aes, + usm_sha512_priv_aes ]. user_tests_cases() -> @@ -311,6 +329,17 @@ event_tests_cases() -> report ]. +v3_cases() -> + [ + simple_v3_exchange_md5, + simple_v3_exchange_sha, + simple_v3_exchange_sha224, + simple_v3_exchange_sha256, + simple_v3_exchange_sha384, + simple_v3_exchange_sha512 + ]. + + event_tests_mt_cases() -> event_tests_cases(). tickets_cases() -> @@ -438,17 +467,19 @@ init_per_group2(all = GroupName, Config) -> ?LIB:init_group_top_dir(GroupName, Config); init_per_group2(request_tests_mt = GroupName, Config) -> ?LIB:init_group_top_dir( - GroupName, + GroupName, [{manager_net_if_module, snmpm_net_if_mt} | Config]); init_per_group2(event_tests_mt = GroupName, Config) -> ?LIB:init_group_top_dir( - GroupName, + GroupName, [{manager_net_if_module, snmpm_net_if_mt} | Config]); init_per_group2(ipv6_mt = GroupName, Config) -> init_per_group_ipv6(GroupName, [{manager_net_if_module, snmpm_net_if_mt} | Config]); -init_per_group2(ipv6 = GroupName, Config) -> - init_per_group_ipv6(GroupName, Config); +init_per_group2(ipv6 = GroupName, Config) -> + init_per_group_ipv6(GroupName, Config); +%% init_per_group2(v3 = GroupName, Config) -> +%% ?LIB:init_group_top_dir(GroupName, Config); init_per_group2(GroupName, Config) -> ?LIB:init_group_top_dir(GroupName, Config). @@ -602,6 +633,18 @@ init_per_testcase2(Case, Config) -> ?DBG("init [~w] Nodes [2]: ~p", [Case, erlang:nodes()]), Conf2. +init_per_testcase3(simple_v3_exchange_md5 = _Case, Config) -> + init_v3_testcase([{auth_alg, md5} | Config]); +init_per_testcase3(simple_v3_exchange_sha = _Case, Config) -> + init_v3_testcase([{auth_alg, sha} | Config]); +init_per_testcase3(simple_v3_exchange_sha224 = _Case, Config) -> + init_v3_testcase([{auth_alg, sha224} | Config]); +init_per_testcase3(simple_v3_exchange_sha256 = _Case, Config) -> + init_v3_testcase([{auth_alg, sha256} | Config]); +init_per_testcase3(simple_v3_exchange_sha384 = _Case, Config) -> + init_v3_testcase([{auth_alg, sha384} | Config]); +init_per_testcase3(simple_v3_exchange_sha512 = _Case, Config) -> + init_v3_testcase([{auth_alg, sha512} | Config]); init_per_testcase3(Case, Config) -> ApiCases02 = [ @@ -707,6 +750,34 @@ init_per_testcase3(Case, Config) -> Config end. +init_v3_testcase(Conf) -> + Conf2 = init_v3_agent(Conf), + Conf3 = try init_v3_manager(Conf2) + catch AC:AE:AS -> + %% Ouch we need to clean up: + %% The init_agent starts an agent node! + ?IPRINT("init_v3_testcase -> failed init v3 manager:" + "~n AC: ~p" + "~n AE: ~p" + "~n AS: ~p", [AC, AE, AS]), + init_per_testcase_fail_agent_cleanup(Conf), + throw({skip, {manager_init_failed, AC, AE}}) + end, + Conf4 = try init_mgr_v3_user(Conf3) + catch MC:ME:MS -> + %% Ouch we need to clean up: + %% The init_agent starts an agent node! + %% The init_magager starts an manager node! + ?IPRINT("init_v3_testcase -> failed init v3 manager user:" + "~n MC: ~p" + "~n ME: ~p" + "~n MS: ~p", [MC, ME, MS]), + init_per_testcase_fail_manager_cleanup(Conf2), + init_per_testcase_fail_agent_cleanup(Conf2), + throw({skip, {manager_user_init_failed, MC, ME}}) + end, + init_mgr_v3_user_data(Conf4). + init_per_testcase_fail_manager_cleanup(Conf) -> (catch fin_manager(Conf)). @@ -726,11 +797,22 @@ end_per_testcase(Case, Config) when is_list(Config) -> Conf2 = end_per_testcase2(Case, Conf1), ?IPRINT("end_per_testcase -> done with" - "~n Condif: ~p" + "~n Config: ~p" "~n Nodes: ~p", [Conf2, erlang:nodes()]), Conf2. +end_per_testcase2(Case, Config) + when ((Case =:= simple_v3_exchange_md5) orelse + (Case =:= simple_v3_exchange_sha) orelse + (Case =:= simple_v3_exchange_sha224) orelse + (Case =:= simple_v3_exchange_sha256) orelse + (Case =:= simple_v3_exchange_sha384) orelse + (Case =:= simple_v3_exchange_sha512)) -> + Conf1 = fin_mgr_user_data2(Config), + Conf2 = fin_mgr_user(Conf1), + Conf3 = fin_v3_manager(Conf2), + fin_v3_agent(Conf3); end_per_testcase2(Case, Config) -> ApiCases02 = [ @@ -1530,7 +1612,7 @@ usm_priv_aes(Config) when is_list(Config) -> ?SLEEP(1000), % Give it time to settle ok end, - Case = fun(_) -> do_usm_priv_aes(Config) end, + Case = fun(_) -> do_usm_priv_aes(sha, Config) end, Post = fun(_) -> ?IPRINT("try stop manager"), ok = snmpm:stop(), @@ -1539,9 +1621,150 @@ usm_priv_aes(Config) when is_list(Config) -> end, ?TC_TRY(usm_priv_aes, Pre, Case, Post). -do_usm_priv_aes(Config) -> + +%%====================================================================== + +usm_sha224_priv_aes(suite) -> []; +usm_sha224_priv_aes(Config) when is_list(Config) -> + Pre = fun() -> + SCO = ?config(socket_create_opts, Config), + ConfDir = ?config(manager_conf_dir, Config), + DbDir = ?config(manager_db_dir, Config), + + write_manager_conf(ConfDir), + + Opts = [{server, [{verbosity, trace}]}, + {net_if, [{verbosity, trace}, {options, SCO}]}, + {note_store, [{verbosity, trace}]}, + {config, [{verbosity, trace}, + {dir, ConfDir}, + {db_dir, DbDir}]}], + + ?IPRINT("try starting manager"), + ok = snmpm:start(Opts), + ?SLEEP(1000), % Give it time to settle + ok + end, + Case = fun(_) -> do_usm_priv_aes(sha224, Config) end, + Post = fun(_) -> + ?IPRINT("try stop manager"), + ok = snmpm:stop(), + ?SLEEP(1000), % Give it time to settle + ok + end, + ?TC_TRY(usm_sha224_priv_aes, Pre, Case, Post). + + +%%====================================================================== + +usm_sha256_priv_aes(suite) -> []; +usm_sha256_priv_aes(Config) when is_list(Config) -> + Pre = fun() -> + SCO = ?config(socket_create_opts, Config), + ConfDir = ?config(manager_conf_dir, Config), + DbDir = ?config(manager_db_dir, Config), + + write_manager_conf(ConfDir), + + Opts = [{server, [{verbosity, trace}]}, + {net_if, [{verbosity, trace}, {options, SCO}]}, + {note_store, [{verbosity, trace}]}, + {config, [{verbosity, trace}, + {dir, ConfDir}, + {db_dir, DbDir}]}], + + ?IPRINT("try starting manager"), + ok = snmpm:start(Opts), + ?SLEEP(1000), % Give it time to settle + ok + end, + Case = fun(_) -> do_usm_priv_aes(sha256, Config) end, + Post = fun(_) -> + ?IPRINT("try stop manager"), + ok = snmpm:stop(), + ?SLEEP(1000), % Give it time to settle + ok + end, + ?TC_TRY(usm_sha256_priv_aes, Pre, Case, Post). + + +%%====================================================================== + +usm_sha384_priv_aes(suite) -> []; +usm_sha384_priv_aes(Config) when is_list(Config) -> + Pre = fun() -> + SCO = ?config(socket_create_opts, Config), + ConfDir = ?config(manager_conf_dir, Config), + DbDir = ?config(manager_db_dir, Config), + + write_manager_conf(ConfDir), + + Opts = [{server, [{verbosity, trace}]}, + {net_if, [{verbosity, trace}, {options, SCO}]}, + {note_store, [{verbosity, trace}]}, + {config, [{verbosity, trace}, + {dir, ConfDir}, + {db_dir, DbDir}]}], + + ?IPRINT("try starting manager"), + ok = snmpm:start(Opts), + ?SLEEP(1000), % Give it time to settle + ok + end, + Case = fun(_) -> do_usm_priv_aes(sha384, Config) end, + Post = fun(_) -> + ?IPRINT("try stop manager"), + ok = snmpm:stop(), + ?SLEEP(1000), % Give it time to settle + ok + end, + ?TC_TRY(usm_sha384_priv_aes, Pre, Case, Post). + + +%%====================================================================== + +usm_sha512_priv_aes(suite) -> []; +usm_sha512_priv_aes(Config) when is_list(Config) -> + Pre = fun() -> + SCO = ?config(socket_create_opts, Config), + ConfDir = ?config(manager_conf_dir, Config), + DbDir = ?config(manager_db_dir, Config), + + write_manager_conf(ConfDir), + + Opts = [{server, [{verbosity, trace}]}, + {net_if, [{verbosity, trace}, {options, SCO}]}, + {note_store, [{verbosity, trace}]}, + {config, [{verbosity, trace}, + {dir, ConfDir}, + {db_dir, DbDir}]}], + + ?IPRINT("try starting manager"), + ok = snmpm:start(Opts), + ?SLEEP(1000), % Give it time to settle + ok + end, + Case = fun(_) -> do_usm_priv_aes(sha512, Config) end, + Post = fun(_) -> + ?IPRINT("try stop manager"), + ok = snmpm:stop(), + ?SLEEP(1000), % Give it time to settle + ok + end, + ?TC_TRY(usm_sha512_priv_aes, Pre, Case, Post). + + +select_auth_proto(md5) -> usmHMACMD5AuthProtocol; +select_auth_proto(sha) -> usmHMACSHAAuthProtocol; +select_auth_proto(sha224) -> usmHMAC128SHA224AuthProtocol; +select_auth_proto(sha256) -> usmHMAC192SHA256AuthProtocol; +select_auth_proto(sha384) -> usmHMAC256SHA384AuthProtocol; +select_auth_proto(sha512) -> usmHMAC384SHA512AuthProtocol. + +do_usm_priv_aes(AuthAlg, Config) -> ?IPRINT("starting with Config: " - "~n ~p", [Config]), + "~n AuthAlg: ~p" + "~n Config: ~p", [AuthAlg, Config]), put(sname, "TC[usm-priv-aes]"), put(verbosity, trace), @@ -1552,13 +1775,13 @@ do_usm_priv_aes(Config) -> SecName = "v3_user", AuthPass = "authpass", AuthKey = - snmp:passwd2localized_key(sha, AuthPass, EngineID), + snmp:passwd2localized_key(AuthAlg, AuthPass, EngineID), PrivPass = "privpass", PrivKey = snmp:passwd2localized_key(md5, PrivPass, EngineID), Credentials = - [ {auth, usmHMACSHAAuthProtocol}, + [ {auth, select_auth_proto(AuthAlg)}, {auth_key, AuthKey}, {priv, usmAesCfb128Protocol}, {priv_key, PrivKey} @@ -3394,6 +3617,75 @@ simple_async_get_bulk3_cbp_perm(Config) when is_list(Config) -> %%====================================================================== +simple_v3_exchange_md5(doc) -> + ["Simple message exchange using v3 (MD5) messages"]; +simple_v3_exchange_md5(suite) -> []; +simple_v3_exchange_md5(Config) when is_list(Config) -> + simple_v3_exchange(Config). + +simple_v3_exchange_sha(doc) -> + ["Simple message exchange using v3 (SHA) messages"]; +simple_v3_exchange_sha(suite) -> []; +simple_v3_exchange_sha(Config) when is_list(Config) -> + simple_v3_exchange(Config). + +simple_v3_exchange_sha224(doc) -> + ["Simple message exchange using v3 (SHA224) messages"]; +simple_v3_exchange_sha224(suite) -> []; +simple_v3_exchange_sha224(Config) when is_list(Config) -> + simple_v3_exchange(Config). + +simple_v3_exchange_sha256(doc) -> + ["Simple message exchange using v3 (SHA256) messages"]; +simple_v3_exchange_sha256(suite) -> []; +simple_v3_exchange_sha256(Config) when is_list(Config) -> + simple_v3_exchange(Config). + +simple_v3_exchange_sha384(doc) -> + ["Simple message exchange using v3 (SHA384) messages"]; +simple_v3_exchange_sha384(suite) -> []; +simple_v3_exchange_sha384(Config) when is_list(Config) -> + simple_v3_exchange(Config). + +simple_v3_exchange_sha512(doc) -> + ["Simple message exchange using v3 (SHA512) messages"]; +simple_v3_exchange_sha512(suite) -> []; +simple_v3_exchange_sha512(Config) when is_list(Config) -> + simple_v3_exchange(Config). + +%% This difference between these test cases are handled in the +%% init_per_testcase function (basically auth algorithm). +simple_v3_exchange(Config) when is_list(Config) -> + ?IPRINT("starting with Config: " + "~n ~p", [Config]), + + Node = ?config(manager_node, Config), + TargetName = ?config(manager_agent_target_name, Config), + + Oids1 = [?sysDescr_instance], + ?IPRINT("initial try get (expect failure and a report)"), + {error, timeout} = mgr_user_sync_get2(Node, TargetName, Oids1, []), + receive + {async_event, TargetName, {report, {noError, 0, [#varbind{oid = ?usmStatsNotInTimeWindows_instance}]}} = Report1} -> + ?IPRINT("received expected report: " + "~n ~p", [Report1]), + ok + end, + ?IPRINT("second try get (expect success)"), + {ok, Reply2, _} = mgr_user_sync_get2(Node, TargetName, Oids1, []), + case Reply2 of + {noError, 0, [#varbind{oid = ?sysDescr_instance}]} -> + ?IPRINT("expected get success"), + display_log(Config), + ok; + X -> + {error, X} + end. + + + +%%====================================================================== + discovery(suite) -> []; discovery(Config) when is_list(Config) -> ?SKIP(not_yet_implemented). @@ -4800,15 +5092,15 @@ init_manager(AutoInform, Config) -> %% -- %% Start and initiate crypto on manager node %% - + ?line ok = init_crypto(Node), - + %% %% Write manager config %% - + ?line ok = write_manager_config(Config), - + IRB = case AutoInform of true -> auto; @@ -4837,6 +5129,67 @@ init_manager(AutoInform, Config) -> ?FAIL({failed_starting_manager, C, E, S}) end. +init_v3_manager(Config) -> + + ?IPRINT("init_v3_manager -> entry with" + "~n Config: ~p", [Config]), + + %% -- + %% Start node + %% + + %% ?line Node = self(), + ?line Node = start_unique_manager_node(), + + %% The point with this (try catch block) is to be + %% able to do some cleanup in case we fail to + %% start some of the apps. That is, if we fail to + %% start the apps (mnesia, crypto and snmp agent) + %% we stop the (agent) node! + + try + begin + + %% -- + %% Start and initiate crypto on manager node + %% + + ?line ok = init_crypto(Node), + + %% + %% Write manager config + %% + + %% DomainType is just a "dummy" arg to make the + %% function choose transportDomainUdpIpv4 as transport domain. + ?line ok = write_manager_config(transport, Config), + + IRB = auto, + Conf = [{manager_node, Node}, {irb, IRB} | Config], + Vsns = [v3], + start_manager(Node, Vsns, Conf) + end + catch + C:{suite_failed, Reason, _M, _L} = E:S when (C =:= exit) -> + ?EPRINT("Failure during manager start (suite-failed):" + "~n Reason: ~p" + "~n StackTrace: ~p", [Reason, S]), + %% And now, *try* to cleanup + (catch stop_node(Node)), + erlang:raise(C, E, S); + C:E:S -> + ?EPRINT("Failure during manager start: " + "~n Error Class: ~p" + "~n Error: ~p" + "~n StackTrace: ~p", [C, E, S]), + %% And now, *try* to cleanup + (catch stop_node(Node)), + ?FAIL({failed_starting_manager, C, E, S}) + end. + +fin_v3_manager(Config) -> + fin_manager(Config). + fin_manager(Config) -> Node = ?config(manager_node, Config), StopMgrRes = stop_manager(Node), @@ -4876,35 +5229,35 @@ init_agent(Config) -> try begin - + %% -- %% Start and initiate mnesia on agent node %% - + ?line ok = init_mnesia(Node, Dir, ?config(mnesia_debug, Config)), - - + + %% -- %% Start and initiate crypto on agent node %% - + ?line ok = init_crypto(Node), - - + + %% %% Write agent config %% - + Vsns = [v1,v2], ?line ok = write_agent_config(Vsns, Config), - + Conf = [{agent_node, Node}, {mib_dir, MibDir} | Config], - + %% %% Start the agent %% - + start_agent(Node, Vsns, Conf) end catch @@ -4924,7 +5277,75 @@ init_agent(Config) -> (catch stop_node(Node)), ?FAIL({failed_starting_agent, C, E, S}) end. - + + +init_v3_agent(Config) -> + ?IPRINT("init_v3_agent -> entry with" + "~n Config: ~p", [Config]), + + %% -- + %% Retrieve some dir's + %% + _Dir = ?config(agent_dir, Config), + MibDir = ?config(mib_dir, Config), + + %% -- + %% Start node + %% + + %% ?line Node = self(), + ?line Node = start_unique_agent_node(), + + %% The point with this (try catch block) is to be + %% able to do some cleanup in case we fail to + %% start some of the apps. That is, if we fail to + %% start the apps (mnesia, crypto and snmp agent) + %% we stop the (agent) node! + + try + begin + + %% -- + %% Start and initiate crypto on agent node + %% + + ?line ok = init_crypto(Node), + + + %% + %% Write agent config + %% + + Vsns = [v3], + ?line ok = write_agent_config(Vsns, Config), + + Conf = [{agent_node, Node}, + {mib_dir, MibDir} | Config], + + %% + %% Start the agent + %% + + start_agent(Node, Vsns, Conf) + end + catch + C:{suite_failed, Reason, _M, _L} = E:S when (C =:= exit) -> + ?EPRINT("Failure during agent start (suite-failed):" + "~n Reason: ~p" + "~n StackTrace: ~p", [Reason, S]), + %% And now, *try* to cleanup + (catch stop_node(Node)), + erlang:raise(C, E, S); + C:E:S -> + ?EPRINT("Failure during agent start: " + "~n Error Class: ~p" + "~n Error: ~p" + "~n StackTrace: ~p", [C, E, S]), + %% And now, *try* to cleanup + (catch stop_node(Node)), + ?FAIL({failed_starting_agent, C, E, S}) + end. + fin_agent(Config) -> Node = ?config(agent_node, Config), @@ -4940,6 +5361,18 @@ fin_agent(Config) -> [Node, StopAgentRes, StopCryptoRes, StopMnesiaRes, StopNode]), Config. +fin_v3_agent(Config) -> + Node = ?config(agent_node, Config), + StopAgentRes = stop_agent(Node), + StopCryptoRes = fin_crypto(Node), + StopNode = stop_node(Node), + ?IPRINT("fin_agent -> stop apps and (agent node ~p) node results: " + "~n SNMP Agent: ~p" + "~n Crypto: ~p" + "~n Node: ~p", + [Node, StopAgentRes, StopCryptoRes, StopNode]), + Config. + init_mnesia(Node, Dir, MnesiaDebug) when ((MnesiaDebug =/= none) andalso (MnesiaDebug =/= debug) andalso (MnesiaDebug =/= trace)) -> @@ -5097,24 +5530,21 @@ mgr_info(Node) -> mgr_sys_info(Node) -> rcall(Node, snmpm_config, system_info, []). -%% mgr_register_user(Node, Id, Data) -> -%% mgr_register_user(Node, Id, ?MODULE, Data). - mgr_register_user(Node, Id, Mod, Data) when is_atom(Mod) -> rcall(Node, snmpm, register_user, [Id, Mod, Data]). +mgr_register_usm_user(Node, EngineID, SecName, Credentials) + when (Node =:= node()) -> + snmpm:register_usm_user(EngineID, SecName, Credentials); +mgr_register_usm_user(Node, EngineID, SecName, Credentials) -> + rcall(Node, snmpm, register_usm_user, [EngineID, SecName, Credentials]). + mgr_unregister_user(Node, Id) -> rcall(Node, snmpm, unregister_user, [Id]). mgr_which_users(Node) -> rcall(Node, snmpm, which_users, []). -%% mgr_register_agent(Node, Id) -> -%% mgr_register_agent(Node, Id, []). - -%% mgr_register_agent(Node, Id, Conf) when is_list(Conf) -> -%% mgr_register_agent(Node, Id, 5000, Conf). - mgr_register_agent(Node, Id, Port, Conf) when is_integer(Port) andalso is_list(Conf) -> Localhost = snmp_test_lib:localhost(), @@ -5127,9 +5557,6 @@ mgr_register_agent(Node, Id, Addr, Port, Conf) when is_integer(Port) andalso is_list(Conf) -> rcall(Node, snmpm, register_agent, [Id, Addr, Port, Conf]). -%% mgr_unregister_agent(Node, Id) -> -%% mgr_unregister_agent(Node, Id, 4000). - mgr_unregister_agent(Node, Id, Port) when is_integer(Port) -> Localhost = snmp_test_lib:localhost(), rcall(Node, snmpm, unregister_agent, [Id, Localhost, Port]); @@ -5208,8 +5635,8 @@ delete_tables(Node, Tabs) -> %% -- Misc manager user wrapper functions -- init_mgr_user(Conf) -> - ?DBG("init_mgr_user -> entry with" - "~n Conf: ~p", [Conf]), + ?IPRINT("init_mgr_user -> entry with" + "~n Conf: ~p", [Conf]), Node = ?config(manager_node, Conf), %% UserId = ?config(user_id, Conf), @@ -5220,6 +5647,57 @@ init_mgr_user(Conf) -> [{user_pid, User} | Conf]. +init_mgr_v3_user(Conf) -> + ?IPRINT("init_mgr_v3_user -> entry with" + "~n Conf: ~p", [Conf]), + + Node = ?config(manager_node, Conf), + %% UserId = ?config(user_id, Conf), + + ?line {ok, User} = mgr_user_start(Node), + ?IPRINT("start_mgr_v3_user -> User: ~p", [User]), + link(User), + + EngineID = "agentEngine", + AuthAlg = ?config(auth_alg, Conf), + SecName = select_secname_from_authalg(AuthAlg),% "authSHA224", + AuthKey = select_authkey_from_authalg(AuthAlg),% "passwd_sha224xxxxxxxxxxxxxxx", + Credentials = + [ {auth, select_auth_proto(AuthAlg)}, + {auth_key, AuthKey} + ], + ?line ok = mgr_register_usm_user(Node, EngineID, SecName, Credentials), + + ?IPRINT("start_mgr_v3_user -> done"), + [{user_pid, User} | Conf]. + +select_secname_from_authalg(md5) -> + "authMD5"; +select_secname_from_authalg(sha) -> + "authSHA"; +select_secname_from_authalg(sha224) -> + "authSHA224"; +select_secname_from_authalg(sha256) -> + "authSHA256"; +select_secname_from_authalg(sha384) -> + "authSHA384"; +select_secname_from_authalg(sha512) -> + "authSHA512". + +select_authkey_from_authalg(md5) -> + "passwd_md5xxxxxx"; +select_authkey_from_authalg(sha) -> + "passwd_shaxxxxxxxxxx"; +select_authkey_from_authalg(sha224) -> + "passwd_sha224xxxxxxxxxxxxxxx"; +select_authkey_from_authalg(sha256) -> + "passwd_sha256xxxxxxxxxxxxxxxxxxx"; +select_authkey_from_authalg(sha384) -> + "passwd_sha384xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"; +select_authkey_from_authalg(sha512) -> + "passwd_sha512xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx". + + fin_mgr_user(Conf) -> User = ?config(user_pid, Conf), unlink(User), @@ -5307,6 +5785,53 @@ init_mgr_user_data2(Conf) -> ?DBG("Updated agent config: ~n~p", [_AgentConf]), Conf. +init_mgr_v3_user_data(Conf) -> + ?IPRINT("init_mgr_v3_user_data -> entry with" + "~n Conf: ~p", [Conf]), + Node = ?config(manager_node, Conf), + TargetName = ?config(manager_agent_target_name, Conf), + IpFamily = ?config(ipfamily, Conf), + Ip = ?config(ip, Conf), + Port = ?AGENT_PORT, + ?line ok = + case IpFamily of + inet -> + mgr_user_register_agent( + Node, TargetName, + [{tdomain, transportDomainUdpIpv4}, + {taddress, {Ip, Port}}, + {engine_id, "agentEngine"}, + {version, v3}]); + inet6 -> + mgr_user_register_agent( + Node, TargetName, + [{tdomain, transportDomainUdpIpv6}, + {taddress, {Ip, Port}}, + {engine_id, "agentEngine"}, + {version, v3}]) + end, + _Agents = mgr_user_which_own_agents(Node), + ?IPRINT("Own agents: ~p", [_Agents]), + + ?line {ok, _DefAgentConf} = mgr_user_agent_info(Node, TargetName, all), + ?IPRINT("Default agent config: ~n~p", [_DefAgentConf]), + + ?line ok = mgr_user_update_agent_info(Node, TargetName, + community, "all-rights"), + SecName = select_secname_from_authalg(?config(auth_alg, Conf)), + ?line ok = mgr_user_update_agent_info(Node, TargetName, + sec_name, SecName), + ?line ok = mgr_user_update_agent_info(Node, TargetName, + sec_level, authNoPriv), + ?line ok = mgr_user_update_agent_info(Node, TargetName, + sec_model, usm), + ?line ok = mgr_user_update_agent_info(Node, TargetName, + max_message_size, 1024), + + ?line {ok, _AgentConf} = mgr_user_agent_info(Node, TargetName, all), + ?IPRINT("Updated agent config: ~n~p", [_AgentConf]), + Conf. + fin_mgr_user_data1(Conf) -> Node = ?config(manager_node, Conf), TargetName = ?config(manager_agent_target_name, Conf), @@ -5329,18 +5854,10 @@ mgr_user_start(Node, Id) -> mgr_user_stop(Node) -> rcall(Node, snmp_manager_user, stop, []). -%% mgr_user_register_agent(Node) -> -%% mgr_user_register_agent(Node, ?LOCALHOST(), ?AGENT_PORT, []). -%% mgr_user_register_agent(Node, TargetName) when is_list(TargetName) -> -%% mgr_user_register_agent(Node, TargetName, []); -%% mgr_user_register_agent(Node, Addr) -> -%% mgr_user_register_agent(Node, Addr, ?AGENT_PORT, []). mgr_user_register_agent(Node, TargetName, Conf) when is_list(TargetName) andalso is_list(Conf) -> rcall(Node, snmp_manager_user, register_agent, [TargetName, Conf]). -%% mgr_user_unregister_agent(Node) -> -%% mgr_user_unregister_agent(Node, ?LOCALHOST(), ?AGENT_PORT). mgr_user_unregister_agent(Node, TargetName) when is_list(TargetName) -> rcall(Node, snmp_manager_user, unregister_agent, [TargetName]). @@ -5348,93 +5865,45 @@ mgr_user_agent_info(Node, TargetName, Item) when is_list(TargetName) andalso is_atom(Item) -> rcall(Node, snmp_manager_user, agent_info, [TargetName, Item]). -%% mgr_user_update_agent_info(Node, Item, Val) when atom(Item) -> -%% mgr_user_update_agent_info(Node, ?LOCALHOST(), ?AGENT_PORT, Item, Val). mgr_user_update_agent_info(Node, TargetName, Item, Val) when is_list(TargetName) andalso is_atom(Item) -> rcall(Node, snmp_manager_user, update_agent_info, [TargetName, Item, Val]). -%% mgr_user_which_all_agents(Node) -> -%% rcall(Node, snmp_manager_user, which_all_agents, []). - mgr_user_which_own_agents(Node) -> rcall(Node, snmp_manager_user, which_own_agents, []). mgr_user_load_mib(Node, Mib) -> rcall(Node, snmp_manager_user, load_mib, [Mib]). -%% mgr_user_sync_get(Node, Oids) -> -%% mgr_user_sync_get(Node, ?LOCALHOST(), ?AGENT_PORT, Oids). -%% mgr_user_sync_get(Node, TargetName, Oids) when is_list(TargetName) -> -%% rcall(Node, snmp_manager_user, sync_get, [TargetName, Oids]). +%% mgr_user_sync_get2(Node, TargetName, Oids) when is_list(TargetName) -> +%% mgr_user_sync_get2(Node, TargetName, Oids, []). mgr_user_sync_get2(Node, TargetName, Oids, SendOpts) when is_list(TargetName) -> rcall(Node, snmp_manager_user, sync_get2, [TargetName, Oids, SendOpts]). -%% mgr_user_async_get(Node, Oids) -> -%% mgr_user_async_get(Node, ?LOCALHOST(), ?AGENT_PORT, Oids). -%% mgr_user_async_get(Node, TargetName, Oids) when is_list(TargetName) -> -%% rcall(Node, snmp_manager_user, async_get, [TargetName, Oids]). - mgr_user_async_get2(Node, TargetName, Oids, SendOpts) when is_list(TargetName) -> rcall(Node, snmp_manager_user, async_get2, [TargetName, Oids, SendOpts]). -%% mgr_user_sync_get_next(Node, Oids) -> -%% mgr_user_sync_get_next(Node, ?LOCALHOST(), ?AGENT_PORT, Oids). -%% mgr_user_sync_get_next(Node, TargetName, Oids) when is_list(TargetName) -> -%% rcall(Node, snmp_manager_user, sync_get_next, [TargetName, Oids]). - mgr_user_sync_get_next2(Node, TargetName, Oids, SendOpts) when is_list(TargetName) -> rcall(Node, snmp_manager_user, sync_get_next2, [TargetName, Oids, SendOpts]). -%% mgr_user_async_get_next(Node, Oids) -> -%% mgr_user_async_get_next(Node, ?LOCALHOST(), ?AGENT_PORT, Oids). -%% mgr_user_async_get_next(Node, TargetName, Oids) when is_list(TargetName) -> -%% rcall(Node, snmp_manager_user, async_get_next, [TargetName, Oids]). - mgr_user_async_get_next2(Node, TargetName, Oids, SendOpts) when is_list(TargetName) -> rcall(Node, snmp_manager_user, async_get_next2, [TargetName, Oids, SendOpts]). -%% mgr_user_sync_set(Node, VAV) -> -%% mgr_user_sync_set(Node, ?LOCALHOST(), ?AGENT_PORT, VAV). -%% mgr_user_sync_set(Node, TargetName, VAV) when is_list(TargetName) -> -%% rcall(Node, snmp_manager_user, sync_set, [TargetName, VAV]). - mgr_user_sync_set2(Node, TargetName, VAV, SendOpts) when is_list(TargetName) -> rcall(Node, snmp_manager_user, sync_set2, [TargetName, VAV, SendOpts]). -%% mgr_user_async_set(Node, VAV) -> -%% mgr_user_async_set(Node, ?LOCALHOST(), ?AGENT_PORT, VAV). -%% mgr_user_async_set(Node, TargetName, VAV) when is_list(TargetName) -> -%% rcall(Node, snmp_manager_user, async_set, [TargetName, VAV]). - mgr_user_async_set2(Node, TargetName, VAV, SendOpts) when is_list(TargetName) -> rcall(Node, snmp_manager_user, async_set2, [TargetName, VAV, SendOpts]). -%% mgr_user_sync_get_bulk(Node, NonRep, MaxRep, Oids) -> -%% mgr_user_sync_get_bulk(Node, ?LOCALHOST(), ?AGENT_PORT, -%% NonRep, MaxRep, Oids). -%% mgr_user_sync_get_bulk(Node, TargetName, NonRep, MaxRep, Oids) -%% when is_list(TargetName) -> -%% rcall(Node, snmp_manager_user, sync_get_bulk, -%% [TargetName, NonRep, MaxRep, Oids]). - mgr_user_sync_get_bulk2(Node, TargetName, NonRep, MaxRep, Oids, SendOpts) when is_list(TargetName) -> rcall(Node, snmp_manager_user, sync_get_bulk2, [TargetName, NonRep, MaxRep, Oids, SendOpts]). -%% mgr_user_async_get_bulk(Node, NonRep, MaxRep, Oids) -> -%% mgr_user_async_get_bulk(Node, ?LOCALHOST(), ?AGENT_PORT, -%% NonRep, MaxRep, Oids). -%% mgr_user_async_get_bulk(Node, TargetName, NonRep, MaxRep, Oids) -%% when is_list(TargetName) -> -%% rcall(Node, snmp_manager_user, async_get_bulk, -%% [TargetName, NonRep, MaxRep, Oids]). - mgr_user_async_get_bulk2(Node, TargetName, NonRep, MaxRep, Oids, SendOpts) when is_list(TargetName) -> rcall(Node, snmp_manager_user, async_get_bulk2, @@ -5542,7 +6011,8 @@ start_agent(Node, Vsns, Conf0, _Opts) -> MSV = get_opt(agent_mib_server_verbosity, Conf0, info), NSV = get_opt(agent_note_store_verbosity, Conf0, info), SSV = get_opt(agent_symbolic_store_verbosity, Conf0, info), - NIV = get_opt(agent_net_if_verbosity, Conf0, log), + %% NIV = get_opt(agent_net_if_verbosity, Conf0, log), + NIV = get_opt(agent_net_if_verbosity, Conf0, trace), Env = [{versions, Vsns}, {type, master}, @@ -5562,7 +6032,7 @@ start_agent(Node, Vsns, Conf0, _Opts) -> {note_store, [{verbosity, NSV}]}, {stymbolic_store, [{verbosity, SSV}]}, {net_if, [{verbosity, NIV}, - %% On some linux "they" add a 127.0.1.1 or somthing + %% On some linux "they" add a 127.0.1.1 or something %% similar, so if we don't specify bind_to %% we don't know which address will be selected %% (which will cause problems for some test cases). @@ -5588,12 +6058,6 @@ stop_agent(Node) -> agent_load_mib(Node, Mib) -> rcall(Node, snmpa, load_mibs, [[Mib]]). -%% agent_unload_mib(Node, Mib) -> -%% rcall(Node, snmpa, unload_mibs, [[Mib]]). - -%% agent_send_trap(Node, Trap, Community) -> -%% Args = [snmp_master_agent, Trap, Community], -%% rcall(Node, snmpa, send_trap, Args). agent_send_trap(Node, Trap, Community, VBs) -> Args = [snmp_master_agent, Trap, Community, VBs], @@ -5612,18 +6076,28 @@ agent_send_notif(Node, Trap, Recv, Name, VBs) -> agent_info(Node) -> rcall(Node, snmpa, info, []). -%% agent_which_mibs(Node) -> -%% rcall(Node, snmpa, which_mibs, []). - %% -- Misc node operation wrapper functions -- +unique(PreName) -> + list_to_atom(?F("~w_~w", [PreName, erlang:system_time(millisecond)])). + start_agent_node() -> start_node(snmp_agent). +start_unique_agent_node() -> + start_unique_node(snmp_agent). + start_manager_node() -> start_node(snmp_manager). +start_unique_manager_node() -> + start_unique_node(snmp_manager). + +%% Generate a "unique" (node-) name and start a node with this name. +start_unique_node(BaseName) when is_atom(BaseName) -> + start_node(unique(BaseName), false). + start_node(Name) -> start_node(Name, true). start_node(Name, Retry) -> @@ -5675,6 +6149,8 @@ start_node(Name, Retry) -> end. +stop_node(Node) when (Node =:= self()) -> + ok;%?line ?FAIL({stop_own_node, Node}); stop_node(Node) -> case ?STOP_NODE(Node) of true -> @@ -5688,12 +6164,21 @@ stop_node(Node) -> %% -- Misc config wrapper functions -- write_manager_config(Config) -> - Dir = ?config(manager_conf_dir, Config), - Ip = tuple_to_list(?config(ip, Config)), + write_manager_config(default, Config). + +write_manager_config(DomainType, Config) -> + Dir = ?config(manager_conf_dir, Config), + Ip = tuple_to_list(?config(ip, Config)), + %% Note that Addr and Port are actually only Addr and Port + %% when DomainType is default. + %% In all other cases the Addr is TransportDomain and + %% port is {Addr, Port}... {Addr, Port} = case ?config(ipfamily, Config) of - inet -> + inet when (DomainType =:= default) -> {Ip, ?MGR_PORT}; + inet -> + {transportDomainUdpIpv4, {Ip, ?MGR_PORT}}; inet6 -> {transportDomainUdpIpv6, {Ip, ?MGR_PORT}} end, @@ -5712,15 +6197,6 @@ write_manager_conf(Dir) -> [Port, MMS, EngineID])), write_manager_conf(Dir, Str). -%% write_manager_conf(Dir, IP, Port, MMS, EngineID) -> -%% Str = lists:flatten( -%% io_lib:format("{address, ~s}.\n" -%% "{port, ~s}.\n" -%% "{max_message_size, ~s}.\n" -%% "{engine_id, ~s}.\n", -%% [IP, Port, MMS, EngineID])), -%% write_manager_conf(Dir, Str). - write_manager_conf(Dir, Str) -> write_conf_file(Dir, "manager.conf", Str). @@ -5731,7 +6207,7 @@ write_agent_config(Vsns, Conf) -> ?line Domain = case ?config(ipfamily, Conf) of inet -> - snmpUDPDomain; + transportDomainUdpIpv4; inet6 -> transportDomainUdpIpv6 end, @@ -5754,21 +6230,44 @@ update_agent_usm(Vsns, Dir) -> true -> Conf = [{"agentEngine", "all-rights", "all-rights", zeroDotZero, usmNoAuthProtocol, "", "", - usmNoPrivProtocol, "", "", "", "", ""}, + usmNoPrivProtocol, "", "", "", + "", ""}, {"agentEngine", "no-rights", "no-rights", zeroDotZero, usmNoAuthProtocol, "", "", - usmNoPrivProtocol, "", "", "", "", ""}, + usmNoPrivProtocol, "", "", "", + "", ""}, {"agentEngine", "authMD5", "authMD5", zeroDotZero, usmHMACMD5AuthProtocol, "", "", - usmNoPrivProtocol, "", "", "", "passwd_md5xxxxxx", ""}, + usmNoPrivProtocol, "", "", "", + "passwd_md5xxxxxx", ""}, {"agentEngine", "authSHA", "authSHA", zeroDotZero, usmHMACSHAAuthProtocol, "", "", usmNoPrivProtocol, "", "", "", "passwd_shaxxxxxxxxxx", ""}, + {"agentEngine", "authSHA224", "authSHA224", zeroDotZero, + usmHMAC128SHA224AuthProtocol, "", "", + usmNoPrivProtocol, "", "", "", + "passwd_sha224xxxxxxxxxxxxxxx", ""}, + + {"agentEngine", "authSHA256", "authSHA256", zeroDotZero, + usmHMAC192SHA256AuthProtocol, "", "", + usmNoPrivProtocol, "", "", "", + "passwd_sha256xxxxxxxxxxxxxxxxxxx", ""}, + + {"agentEngine", "authSHA384", "authSHA384", zeroDotZero, + usmHMAC256SHA384AuthProtocol, "", "", + usmNoPrivProtocol, "", "", "", + "passwd_sha384xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx", ""}, + + {"agentEngine", "authSHA512", "authSHA512", zeroDotZero, + usmHMAC384SHA512AuthProtocol, "", "", + usmNoPrivProtocol, "", "", "", + "passwd_sha512xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx", ""}, + {"agentEngine", "privDES", "privDES", zeroDotZero, usmHMACSHAAuthProtocol, "", "", usmDESPrivProtocol, "", "", "", @@ -5776,21 +6275,44 @@ update_agent_usm(Vsns, Dir) -> {"mgrEngine", "all-rights", "all-rights", zeroDotZero, usmNoAuthProtocol, "", "", - usmNoPrivProtocol, "", "", "", "", ""}, + usmNoPrivProtocol, "", "", "", + "", ""}, {"mgrEngine", "no-rights", "no-rights", zeroDotZero, usmNoAuthProtocol, "", "", - usmNoPrivProtocol, "", "", "", "", ""}, + usmNoPrivProtocol, "", "", "", + "", ""}, {"mgrEngine", "authMD5", "authMD5", zeroDotZero, usmHMACMD5AuthProtocol, "", "", - usmNoPrivProtocol, "", "", "", "passwd_md5xxxxxx", ""}, + usmNoPrivProtocol, "", "", "", + "passwd_md5xxxxxx", ""}, {"mgrEngine", "authSHA", "authSHA", zeroDotZero, usmHMACSHAAuthProtocol, "", "", usmNoPrivProtocol, "", "", "", "passwd_shaxxxxxxxxxx", ""}, + {"mgrEngine", "authSHA224", "authSHA224", zeroDotZero, + usmHMAC128SHA224AuthProtocol, "", "", + usmNoPrivProtocol, "", "", "", + "passwd_sha224xxxxxxxxxxxxxxx", ""}, + + {"mgrEngine", "authSHA256", "authSHA256", zeroDotZero, + usmHMAC192SHA256AuthProtocol, "", "", + usmNoPrivProtocol, "", "", "", + "passwd_sha256xxxxxxxxxxxxxxxxxxx", ""}, + + {"mgrEngine", "authSHA384", "authSHA384", zeroDotZero, + usmHMAC256SHA384AuthProtocol, "", "", + usmNoPrivProtocol, "", "", "", + "passwd_sha384xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx", ""}, + + {"mgrEngine", "authSHA512", "authSHA512", zeroDotZero, + usmHMAC384SHA512AuthProtocol, "", "", + usmNoPrivProtocol, "", "", "", + "passwd_sha512xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx", ""}, + {"mgrEngine", "privDES", "privDES", zeroDotZero, usmHMACSHAAuthProtocol, "", "", usmDESPrivProtocol, "", "", "", @@ -5807,10 +6329,14 @@ update_agent_community(_, Dir) -> snmp_config:update_agent_community_config(Dir, Conf). update_agent_vacm(_Vsns, Dir) -> - Conf = [{vacmSecurityToGroup, usm, "authMD5", "initial"}, - {vacmSecurityToGroup, usm, "authSHA", "initial"}, - {vacmSecurityToGroup, usm, "privDES", "initial"}, - {vacmSecurityToGroup, usm, "newUser", "initial"}, + Conf = [{vacmSecurityToGroup, usm, "authMD5", "initial"}, + {vacmSecurityToGroup, usm, "authSHA", "initial"}, + {vacmSecurityToGroup, usm, "authSHA224", "initial"}, + {vacmSecurityToGroup, usm, "authSHA256", "initial"}, + {vacmSecurityToGroup, usm, "authSHA384", "initial"}, + {vacmSecurityToGroup, usm, "authSHA512", "initial"}, + {vacmSecurityToGroup, usm, "privDES", "initial"}, + {vacmSecurityToGroup, usm, "newUser", "initial"}, {vacmViewTreeFamily, "internet", ?tDescr_instance, excluded, null}], snmp_config:update_agent_vacm_config(Dir, Conf). @@ -5906,6 +6432,8 @@ get_opt(Key, Opts, Def) -> %% ------ +rcall(Node, Mod, Func, Args) when (Node =:= self()) -> + apply(Mod, Func, Args); rcall(Node, Mod, Func, Args) -> case rpc:call(Node, Mod, Func, Args) of {badrpc, nodedown} -> diff --git a/lib/snmp/test/snmp_manager_config_SUITE.erl b/lib/snmp/test/snmp_manager_config_SUITE.erl index f7f7fd6928..06bae05556 100644 --- a/lib/snmp/test/snmp_manager_config_SUITE.erl +++ b/lib/snmp/test/snmp_manager_config_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2004-2020. All Rights Reserved. +%% Copyright Ericsson AB 2004-2021. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -1574,112 +1574,257 @@ start_with_invalid_usm_conf_file1(Conf) when is_list(Conf) -> config_ensure_not_running(), %% -- - ?IPRINT("[test 51] write usm config file with invalid auth-key (1)"), - Usm51 = setelement(3, Usm0, "usmHMACMD5AuthProtocol"), - write_usm_conf(ConfDir, [Usm51]), - ?line {error, Reason51} = config_start(Opts), - ?IPRINT("start failed (as expected): ~p", [Reason51]), - ?line {failed_check, _, _, _, {invalid_auth_key, _, _}} = Reason51, + ?IPRINT("[test 51.1] write (auth md5) usm config file with invalid auth-key (1)"), + Usm51_1 = setelement(3, Usm0, "usmHMACMD5AuthProtocol"), + write_usm_conf(ConfDir, [Usm51_1]), + ?line {error, Reason51_1} = config_start(Opts), + ?IPRINT("start failed (as expected): ~p", [Reason51_1]), + ?line {failed_check, _, _, _, {invalid_auth_key, _, _}} = Reason51_1, config_ensure_not_running(), %% -- - ?IPRINT("[test 52] write usm config file with invalid auth-key (2)"), - Usm52 = setelement(4, Usm51, "[1,2,3,4,5,6,7,8,9,0,1,2,3,4,5]"), - write_usm_conf(ConfDir, [Usm52]), - ?line {error, Reason52} = config_start(Opts), - ?IPRINT("start failed (as expected): ~p", [Reason52]), - ?line {failed_check, _, _, _, {invalid_auth_key, _, 15}} = Reason52, + ?IPRINT("[test 51.2] write (auth md5) usm config file with invalid auth-key (2)"), + Usm51_2 = setelement(4, Usm51_1, "[1,2,3,4,5,6,7,8,9,0,1,2,3,4,5]"), + write_usm_conf(ConfDir, [Usm51_2]), + ?line {error, Reason51_2} = config_start(Opts), + ?IPRINT("start failed (as expected): ~p", [Reason51_2]), + ?line {failed_check, _, _, _, {invalid_auth_key, _, 15}} = Reason51_2, config_ensure_not_running(), %% -- - ?IPRINT("[test 53] write usm config file with invalid auth-key (3)"), - Usm53 = setelement(4, Usm51, "[1,2,3,4,5,6,7,8,9,0,1,2,3,4,5,6,7]"), - write_usm_conf(ConfDir, [Usm53]), - ?line {error, Reason53} = config_start(Opts), - ?IPRINT("start failed (as expected): ~p", [Reason53]), - ?line {failed_check, _, _, _, {invalid_auth_key, _, 17}} = Reason53, + ?IPRINT("[test 51.3] write (auth md5) usm config file with invalid auth-key (3)"), + Usm51_3 = setelement(4, Usm51_1, "[1,2,3,4,5,6,7,8,9,0,1,2,3,4,5,6,7]"), + write_usm_conf(ConfDir, [Usm51_3]), + ?line {error, Reason51_3} = config_start(Opts), + ?IPRINT("start failed (as expected): ~p", [Reason51_3]), + ?line {failed_check, _, _, _, {invalid_auth_key, _, 17}} = Reason51_3, config_ensure_not_running(), %% -- - ?IPRINT("[test 54] write usm config file with invalid auth-key (4)"), - Usm54 = setelement(4, Usm51, "[1,2,3,4,5,6,7,8,9,0,1,2,3,4,5,kalle]"), - write_usm_conf(ConfDir, [Usm54]), + ?IPRINT("[test 51.4] write (auth md5) usm config file with invalid auth-key (4)"), + Usm51_4 = setelement(4, Usm51_1, "[1,2,3,4,5,6,7,8,9,0,1,2,3,4,5,kalle]"), + write_usm_conf(ConfDir, [Usm51_4]), ?line maybe_start_crypto(), %% Make sure it's started... - ?line {error, Reason54} = config_start(Opts), + ?line {error, Reason51_4} = config_start(Opts), ?line ok = maybe_stop_crypto(), - ?IPRINT("start failed (as expected): ~p", [Reason54]), - ?line {failed_check, _, _, _, {invalid_auth_key, _}} = Reason54, + ?IPRINT("start failed (as expected): ~p", [Reason51_4]), + ?line {failed_check, _, _, _, {invalid_auth_key, _}} = Reason51_4, config_ensure_not_running(), %% -- - ?IPRINT("[test 55] write usm config file with invalid auth-key (5)"), - Usm55 = setelement(4, Usm51, "arne_anka"), - write_usm_conf(ConfDir, [Usm55]), - ?line {error, Reason55} = config_start(Opts), - ?IPRINT("start failed (as expected): ~p", [Reason55]), - ?line {failed_check, _, _, _, {invalid_auth_key, _}} = Reason55, + ?IPRINT("[test 51.5] write (auth md5) usm config file with invalid auth-key (5)"), + Usm51_5 = setelement(4, Usm51_1, "arne_anka"), + write_usm_conf(ConfDir, [Usm51_5]), + ?line {error, Reason51_5} = config_start(Opts), + ?IPRINT("start failed (as expected): ~p", [Reason51_5]), + ?line {failed_check, _, _, _, {invalid_auth_key, _}} = Reason51_5, config_ensure_not_running(), %% -- - ?IPRINT("[test 56] write usm config file with invalid auth-key (6)"), - Usm56 = setelement(4, Usm51, "10101"), - write_usm_conf(ConfDir, [Usm56]), - ?line {error, Reason56} = config_start(Opts), - ?IPRINT("start failed (as expected): ~p", [Reason56]), - ?line {failed_check, _, _, _, {invalid_auth_key, _}} = Reason56, + ?IPRINT("[test 51.6] write (auth md5) usm config file with invalid auth-key (6)"), + Usm51_6 = setelement(4, Usm51_1, "10101"), + write_usm_conf(ConfDir, [Usm51_6]), + ?line {error, Reason51_6} = config_start(Opts), + ?IPRINT("start failed (as expected): ~p", [Reason51_6]), + ?line {failed_check, _, _, _, {invalid_auth_key, _}} = Reason51_6, + config_ensure_not_running(), + + + + %% -- (auth) SHA -- + ?IPRINT("[test 52.1] write (auth sha) usm config file with invalid auth-key (1)"), + Usm52_1 = setelement(3, Usm0, "usmHMACSHAAuthProtocol"), + write_usm_conf(ConfDir, [Usm52_1]), + ?line {error, Reason52_1} = config_start(Opts), + ?IPRINT("start failed (as expected): ~p", [Reason52_1]), + ?line {failed_check, _, _, _, {invalid_auth_key, _, _}} = Reason52_1, + config_ensure_not_running(), + + %% -- + ?IPRINT("[test 52.2] write (auth sha) usm config file with invalid auth-key (2)"), + Usm52_2 = setelement(4, Usm52_1, "[1,2,3,4,5,6,7,8,9,0,1,2,3,4,5,6]"), + write_usm_conf(ConfDir, [Usm52_2]), + ?line {error, Reason52_2} = config_start(Opts), + ?IPRINT("start failed (as expected): ~p", [Reason52_2]), + ?line {failed_check, _, _, _, {invalid_auth_key, _, 16}} = Reason52_2, + config_ensure_not_running(), + + %% -- + ?IPRINT("[test 52.3] write (auth sha) usm config file with invalid auth-key (3)"), + Usm52_3 = setelement(4, Usm52_1, "[1,2,3,4,5,6,7,8,9,0,1,2,3,4,5,6,7,8,9,ka]"), + write_usm_conf(ConfDir, [Usm52_3]), + ?line ok = maybe_start_crypto(), + ?line {error, Reason52_3} = config_start(Opts), + ?line ok = maybe_stop_crypto(), + ?IPRINT("start failed (as expected): ~p", [Reason52_3]), + ?line {failed_check, _, _, _, {invalid_auth_key, _}} = Reason52_3, + config_ensure_not_running(), + + + + %% -- (auth) SHA-224 -- + ?IPRINT("[test 53.1] write (auth sha224) usm config file with invalid auth-key (1)"), + Usm53_1 = setelement(3, Usm0, "usmHMAC128SHA224AuthProtocol"), + write_usm_conf(ConfDir, [Usm53_1]), + ?line {error, Reason53_1} = config_start(Opts), + ?IPRINT("start failed (as expected): ~p", [Reason53_1]), + ?line {failed_check, _, _, _, {invalid_auth_key, _, _}} = Reason53_1, config_ensure_not_running(), %% -- - ?IPRINT("[test 57] write usm config file with invalid auth-key (7)"), - Usm57 = setelement(3, Usm0, "usmHMACSHAAuthProtocol"), - write_usm_conf(ConfDir, [Usm57]), - ?line {error, Reason57} = config_start(Opts), - ?IPRINT("start failed (as expected): ~p", [Reason57]), - ?line {failed_check, _, _, _, {invalid_auth_key, _, _}} = Reason57, + ?IPRINT("[test 53.2] write (auth sha224) usm config file with valid auth-key (2)"), + Usm53_2 = setelement(4, Usm53_1, + "[1,2,3,4,5,6,7,8,9,0,1,2,3,4,5,6,7,8,9,0,1,2,3,4,5,6,7,8]"), + write_usm_conf(ConfDir, [Usm53_2]), + ?line {ok, _} = config_start(Opts), + ?IPRINT("expected start success"), config_ensure_not_running(), %% -- - ?IPRINT("[test 58] write usm config file with invalid auth-key (8)"), - Usm58 = setelement(4, Usm57, "[1,2,3,4,5,6,7,8,9,0,1,2,3,4,5,6]"), - write_usm_conf(ConfDir, [Usm58]), - ?line {error, Reason58} = config_start(Opts), - ?IPRINT("start failed (as expected): ~p", [Reason58]), - ?line {failed_check, _, _, _, {invalid_auth_key, _, 16}} = Reason58, + ?IPRINT("[test 53.3] write (auth sha224) usm config file with invalid auth-key (3)"), + Usm53_3 = setelement(4, Usm53_1, + "[1,2,3,4,5,6,7,8,9,0,1,2,3,4,5,6,7,8,9,0,1,2,3,4,5,6,7]"), + write_usm_conf(ConfDir, [Usm53_3]), + ?line {error, Reason53_3} = config_start(Opts), + ?IPRINT("start failed (as expected): ~p", [Reason53_3]), + ?line {failed_check, _, _, _, {invalid_auth_key, _, 27}} = Reason53_3, config_ensure_not_running(), %% -- - ?IPRINT("[test 59] write usm config file with invalid auth-key (9)"), - Usm59 = setelement(4, Usm57, "[1,2,3,4,5,6,7,8,9,0,1,2,3,4,5,6,7,8,9,ka]"), - write_usm_conf(ConfDir, [Usm59]), + ?IPRINT("[test 53.4] write (auth sha224) usm config file with invalid auth-key (4)"), + Usm53_4 = setelement(4, Usm53_1, + "[1,2,3,4,5,6,7,8,9,0,1,2,3,4,5,6,7,8,9,0,1,2,3,4,5,6,7,ka]"), + write_usm_conf(ConfDir, [Usm53_4]), ?line ok = maybe_start_crypto(), - ?line {error, Reason59} = config_start(Opts), + ?line {error, Reason53_4} = config_start(Opts), ?line ok = maybe_stop_crypto(), - ?IPRINT("start failed (as expected): ~p", [Reason59]), - ?line {failed_check, _, _, _, {invalid_auth_key, _}} = Reason59, + ?IPRINT("start failed (as expected): ~p", [Reason53_4]), + ?line {failed_check, _, _, _, {invalid_auth_key, _}} = Reason53_4, + config_ensure_not_running(), + + + %% -- (auth) SHA-256 -- + ?IPRINT("[test 54.1] write (auth sha256) usm config file with invalid auth-key (1)"), + Usm54_1 = setelement(3, Usm0, "usmHMAC192SHA256AuthProtocol"), + write_usm_conf(ConfDir, [Usm54_1]), + ?line {error, Reason54_1} = config_start(Opts), + ?IPRINT("start failed (as expected): ~p", [Reason54_1]), + ?line {failed_check, _, _, _, {invalid_auth_key, _, _}} = Reason54_1, config_ensure_not_running(), %% -- - %% <CRYPTO-MODIFICATIONS> - %% The crypto application do no longer need to be started - %% explicitly (all of it is as of R14 implemented with NIFs). - case (catch crypto:version()) of - {'EXIT', {undef, _}} -> - ?IPRINT("[test 5A] write usm config file with valid auth-key " - "when crypto not started (10)"), - Usm5A = setelement(4, - Usm57, - "[1,2,3,4,5,6,7,8,9,0,1,2,3,4,5,6,7,8,9,0]"), - write_usm_conf(ConfDir, [Usm5A]), - ?line {error, Reason5A} = config_start(Opts), - ?IPRINT("start failed (as expected): ~p", [Reason5A]), - ?line {failed_check, _, _, _, {unsupported_crypto, _}} = Reason5A, - config_ensure_not_running(); - _ -> - %% This function is only present in version 2.0 or greater. - %% The crypto app no longer needs to be explicitly started - ok - end, - %% </CRYPTO-MODIFICATIONS> + ?IPRINT("[test 54.2] write (auth sha256) usm config file with valid auth-key (2)"), + Usm54_2 = setelement(4, Usm54_1, + "[1,2,3,4,5,6,7,8,9,0,1,2,3,4,5,6,7,8,9,0,1,2,3,4,5,6,7,8,9,0,1,2]"), + write_usm_conf(ConfDir, [Usm54_2]), + ?line {ok, _} = config_start(Opts), + ?IPRINT("expected start success"), + config_ensure_not_running(), + + %% -- + ?IPRINT("[test 54.3] write (auth sha256) usm config file with invalid auth-key (3)"), + Usm54_3 = setelement(4, Usm54_1, + "[1,2,3,4,5,6,7,8,9,0,1,2,3,4,5,6,7,8,9,0,1,2,3,4,5,6,7,8,9,0,1]"), + write_usm_conf(ConfDir, [Usm54_3]), + ?line {error, Reason54_3} = config_start(Opts), + ?IPRINT("start failed (as expected): ~p", [Reason54_3]), + ?line {failed_check, _, _, _, {invalid_auth_key, _, 31}} = Reason54_3, + config_ensure_not_running(), + + %% -- + ?IPRINT("[test 54.4] write (auth sha256) usm config file with invalid auth-key (4)"), + Usm54_4 = setelement(4, Usm54_1, + "[1,2,3,4,5,6,7,8,9,0,1,2,3,4,5,6,7,8,9,0,1,2,3,4,5,6,7,8,9,0,1,ka]"), + write_usm_conf(ConfDir, [Usm54_4]), + ?line ok = maybe_start_crypto(), + ?line {error, Reason54_4} = config_start(Opts), + ?line ok = maybe_stop_crypto(), + ?IPRINT("start failed (as expected): ~p", [Reason54_4]), + ?line {failed_check, _, _, _, {invalid_auth_key, _}} = Reason54_4, + config_ensure_not_running(), + + + %% -- (auth) SHA-384 -- + ?IPRINT("[test 55.1] write (auth sha384) usm config file with invalid auth-key (1)"), + Usm55_1 = setelement(3, Usm0, "usmHMAC256SHA384AuthProtocol"), + write_usm_conf(ConfDir, [Usm55_1]), + ?line {error, Reason55_1} = config_start(Opts), + ?IPRINT("start failed (as expected): ~p", [Reason55_1]), + ?line {failed_check, _, _, _, {invalid_auth_key, _, _}} = Reason55_1, + config_ensure_not_running(), + + %% -- + ?IPRINT("[test 55.2] write (auth sha384) usm config file with valid auth-key (2)"), + Usm55_2 = setelement(4, Usm55_1, + "[1,2,3,4,5,6,7,8,9,0,1,2,3,4,5,6,7,8,9,0,1,2,3,4,5,6,7,8,9,0,1,2,3,4,5,6,7,8,9,0,1,2,3,4,5,6,7,8]"), + write_usm_conf(ConfDir, [Usm55_2]), + ?line {ok, _} = config_start(Opts), + ?IPRINT("expected start success"), + config_ensure_not_running(), + + %% -- + ?IPRINT("[test 55.3] write (auth sha384) usm config file with invalid auth-key (3)"), + Usm55_3 = setelement(4, Usm55_1, + "[1,2,3,4,5,6,7,8,9,0,1,2,3,4,5,6,7,8,9,0,1,2,3,4,5,6,7,8,9,0,1,2,3,4,5,6,7,8,9,0,1,2,3,4,5,6,7]"), + write_usm_conf(ConfDir, [Usm55_3]), + ?line {error, Reason55_3} = config_start(Opts), + ?IPRINT("start failed (as expected): ~p", [Reason55_3]), + ?line {failed_check, _, _, _, {invalid_auth_key, _, 47}} = Reason55_3, + config_ensure_not_running(), + + %% -- + ?IPRINT("[test 55.4] write (auth sha384) usm config file with invalid auth-key (4)"), + Usm55_4 = setelement(4, Usm55_1, + "[1,2,3,4,5,6,7,8,9,0,1,2,3,4,5,6,7,8,9,0,1,2,3,4,5,6,7,8,9,0,1,2,3,4,5,6,7,8,9,0,1,2,3,4,5,6,7,ka]"), + write_usm_conf(ConfDir, [Usm55_4]), + ?line ok = maybe_start_crypto(), + ?line {error, Reason55_4} = config_start(Opts), + ?line ok = maybe_stop_crypto(), + ?IPRINT("start failed (as expected): ~p", [Reason55_4]), + ?line {failed_check, _, _, _, {invalid_auth_key, _}} = Reason55_4, + config_ensure_not_running(), + + + %% -- (auth) SHA-512 -- + ?IPRINT("[test 56.1] write (auth sha512) usm config file with invalid auth-key (1)"), + Usm56_1 = setelement(3, Usm0, "usmHMAC384SHA512AuthProtocol"), + write_usm_conf(ConfDir, [Usm56_1]), + ?line {error, Reason56_1} = config_start(Opts), + ?IPRINT("start failed (as expected): ~p", [Reason56_1]), + ?line {failed_check, _, _, _, {invalid_auth_key, _, _}} = Reason56_1, + config_ensure_not_running(), + + %% -- + ?IPRINT("[test 56.2] write (auth sha512) usm config file with valid auth-key (2)"), + Usm56_2 = setelement(4, Usm56_1, + "[1,2,3,4,5,6,7,8,9,0,1,2,3,4,5,6,7,8,9,0,1,2,3,4,5,6,7,8,9,0,1,2,3,4,5,6,7,8,9,0,1,2,3,4,5,6,7,8,9,0,1,2,3,4,5,6,7,8,9,0,1,2,3,4]"), + write_usm_conf(ConfDir, [Usm56_2]), + ?line {ok, _} = config_start(Opts), + ?IPRINT("expected start success"), + config_ensure_not_running(), + + %% -- + ?IPRINT("[test 56.3] write (auth sha512) usm config file with invalid auth-key (3)"), + Usm56_3 = setelement(4, Usm56_1, + "[1,2,3,4,5,6,7,8,9,0,1,2,3,4,5,6,7,8,9,0,1,2,3,4,5,6,7,8,9,0,1,2,3,4,5,6,7,8,9,0,1,2,3,4,5,6,7,8,9,0,1,2,3,4,5,6,7,8,9,0,1,2,3]"), + write_usm_conf(ConfDir, [Usm56_3]), + ?line {error, Reason56_3} = config_start(Opts), + ?IPRINT("start failed (as expected): ~p", [Reason56_3]), + ?line {failed_check, _, _, _, {invalid_auth_key, _, 63}} = Reason56_3, + config_ensure_not_running(), + + %% -- + ?IPRINT("[test 56.4] write (auth sha512) usm config file with invalid auth-key (4)"), + Usm56_4 = setelement(4, Usm56_1, + "[1,2,3,4,5,6,7,8,9,0,1,2,3,4,5,6,7,8,9,0,1,2,3,4,5,6,7,8,9,0,1,2,3,4,5,6,7,8,9,0,1,2,3,4,5,6,7,8,9,0,1,2,3,4,5,6,7,8,9,0,1,2,3,ka]"), + write_usm_conf(ConfDir, [Usm56_4]), + ?line ok = maybe_start_crypto(), + ?line {error, Reason56_4} = config_start(Opts), + ?line ok = maybe_stop_crypto(), + ?IPRINT("start failed (as expected): ~p", [Reason56_4]), + ?line {failed_check, _, _, _, {invalid_auth_key, _}} = Reason56_4, + config_ensure_not_running(), + %% -- ?IPRINT("[test 61] write usm config file with invalid priv-protocol (1)"), diff --git a/lib/snmp/test/snmp_manager_user.erl b/lib/snmp/test/snmp_manager_user.erl index 5c56ed5c15..cc62551cec 100644 --- a/lib/snmp/test/snmp_manager_user.erl +++ b/lib/snmp/test/snmp_manager_user.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2005-2016. All Rights Reserved. +%% Copyright Ericsson AB 2005-2021. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -39,28 +39,28 @@ %% External exports %%---------------------------------------------------------------------- -export([ - start_link/0, start_link/1, start_link/2, - start/0, start/1, start/2, + start_link/0, start_link/1, start_link/2, + start/0, start/1, start/2, stop/0, - info/0, - system_info/0, + info/0, + system_info/0, simulate_crash/1, - register_agent/2, - unregister_agent/1, - agent_info/2, - update_agent_info/3, - which_all_agents/0, which_own_agents/0, - load_mib/1, unload_mib/1, - sync_get2/3, + register_agent/2, + unregister_agent/1, + agent_info/2, + update_agent_info/3, + which_all_agents/0, which_own_agents/0, + load_mib/1, unload_mib/1, + sync_get2/3, async_get2/3, sync_get_next2/3, async_get_next2/3, - sync_set2/3, - async_set2/3, + sync_set2/3, + async_set2/3, sync_get_bulk2/5, async_get_bulk2/5, - name_to_oid/1, oid_to_name/1, - purify_oid/1 + name_to_oid/1, oid_to_name/1, + purify_oid/1 ]). diff --git a/lib/snmp/test/snmp_manager_user_SUITE.erl b/lib/snmp/test/snmp_manager_user_SUITE.erl index bad746b29a..04a75232b7 100644 --- a/lib/snmp/test/snmp_manager_user_SUITE.erl +++ b/lib/snmp/test/snmp_manager_user_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2004-2020. All Rights Reserved. +%% Copyright Ericsson AB 2004-2021. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -195,6 +195,11 @@ init_per_testcase(Case, Config) when is_list(Config) -> snmp_test_global_sys_monitor:reset_events(), + ?IPRINT("init_per_testcase -> ensure (snmp) manager not running"), + Mod = snmpm_supervisor, + Stopper = fun() -> Mod:stop() end, + ?ENSURE_NOT_RUNNING(Mod, Stopper, 1000), + SuiteTopDir = ?config(snmp_suite_top_dir, Config), CaseTopDir = filename:join(SuiteTopDir, atom_to_list(Case)), ?line ok = file:make_dir(CaseTopDir), @@ -1291,7 +1296,7 @@ which_agents() -> write_manager_conf(Dir) -> - Port = "5000", + Port = "0", MMS = "484", EngineID = "\"mgrEngine\"", Str = lists:flatten( diff --git a/lib/snmp/test/snmp_test_global_sys_monitor.erl b/lib/snmp/test/snmp_test_global_sys_monitor.erl index c77b118d18..1c90fff686 100644 --- a/lib/snmp/test/snmp_test_global_sys_monitor.erl +++ b/lib/snmp/test/snmp_test_global_sys_monitor.erl @@ -22,7 +22,7 @@ -export([start/0, stop/0, reset_events/0, - events/0, + events/0, events/1, log/1]). -export([init/1]). @@ -47,7 +47,10 @@ reset_events() -> call(reset_events, ?TIMEOUT). events() -> - call(events, ?TIMEOUT). + events(?TIMEOUT). + +events(Timeout) when is_integer(Timeout) andalso (Timeout > 0) -> + call(events, Timeout). log(Event) -> cast({node(), Event}). @@ -229,10 +232,10 @@ call(Req, Timeout) when is_integer(Timeout) -> %% This peace of wierdness is because on some machines this call has %% hung (in a call during end_per_testcase, which had a 1 min timeout, %% or if that was the total time for the test case). -%% But because it hung there, we don't really know what where it git stuck. +%% But because it hung there, we don't really know where it got stuck. %% So, by making the call in a tmp process, that we supervise, we can %% keep control. Also, we change the default timeout from infinity to an -%% actual time (16 seconds). +%% actual time (6 seconds). call(Req, Timeout1, Timeout2) -> F = fun() -> Ref = make_ref(), diff --git a/lib/snmp/test/snmp_to_snmpnet_SUITE.erl b/lib/snmp/test/snmp_to_snmpnet_SUITE.erl index aacdcff504..b673d0fc0d 100644 --- a/lib/snmp/test/snmp_to_snmpnet_SUITE.erl +++ b/lib/snmp/test/snmp_to_snmpnet_SUITE.erl @@ -52,7 +52,7 @@ expected(?sysDescr_instance = Oid, get) -> OidStr = oid_str(Oid), - iolist_to_binary([OidStr | " = STRING: \"Erlang SNMP agent\""]). + lists:flatten([OidStr | " = STRING: \"Erlang SNMP agent\""]). %%-------------------------------------------------------------------- @@ -402,8 +402,19 @@ erlang_agent_netsnmp_get(Config) when is_list(Config) -> Expected = expected(Oid, get), try begin - [Expected = snmpget(Oid, Transport, Config) - || Transport <- Transports], + [case snmpget(Oid, Transport, Config) of + Expected -> + ct:pal("Received expected", []), + ok; + "Timeout: " ++ Rest -> + ct:pal("Received unexpected timeout: " + "~n ~s", [Rest]), + throw({skip, Rest}); + Any -> + ct:pal("Received unexpected reponse: " + "~n ~p", [Any]), + exit({unexpected, Any}) + end || Transport <- Transports], ok end catch @@ -513,7 +524,7 @@ snmpget(Oid, Transport, Config) -> ProgHandle -> {_, line, Line} = get_program_output(ProgHandle), stop_program(ProgHandle), - Line + binary_to_list(Line) end. start_snmptrapd(Mibs, Config) -> diff --git a/lib/ssh/doc/src/notes.xml b/lib/ssh/doc/src/notes.xml index 24ef46ba9c..876553e745 100644 --- a/lib/ssh/doc/src/notes.xml +++ b/lib/ssh/doc/src/notes.xml @@ -30,6 +30,24 @@ <file>notes.xml</file> </header> +<section><title>Ssh 4.12.5</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Fixed a race condition in the acceptor loop: if a client + disconnected immediately after the tcp connect, the + server could cease handling connection on that + address:port.</p> + <p> + Own Id: OTP-17764 Aux Id: ERIERL-726 </p> + </item> + </list> + </section> + +</section> + <section><title>Ssh 4.12.4</title> <section><title>Fixed Bugs and Malfunctions</title> @@ -209,6 +227,22 @@ </section> +<section><title>Ssh 4.11.1.4</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + The value of the <c>connect_timeout</c> option is now + used as default value for the negotiation timeout.</p> + <p> + Own Id: OTP-17707 Aux Id: ERIERL-706 </p> + </item> + </list> + </section> + +</section> + <section><title>Ssh 4.11.1.3</title> <section><title>Fixed Bugs and Malfunctions</title> @@ -776,6 +810,22 @@ </section> +<section><title>Ssh 4.9.1.4</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + The value of the <c>connect_timeout</c> option is now + used as default value for the negotiation timeout.</p> + <p> + Own Id: OTP-17707 Aux Id: ERIERL-706 </p> + </item> + </list> + </section> + +</section> + <section><title>Ssh 4.9.1.3</title> <section><title>Fixed Bugs and Malfunctions</title> diff --git a/lib/ssh/doc/src/ssh.xml b/lib/ssh/doc/src/ssh.xml index 377643e69e..a9db9f418b 100644 --- a/lib/ssh/doc/src/ssh.xml +++ b/lib/ssh/doc/src/ssh.xml @@ -1274,7 +1274,9 @@ <p>No channel is started. This is done by calling <seemfa marker="ssh_connection#session_channel/2"> ssh_connection:session_channel/[2, 4]</seemfa>. </p> - <p>The <c>NegotiationTimeout</c> is in milli-seconds. The default value is <c>infinity</c>. + <p>The <c>NegotiationTimeout</c> is in milli-seconds. The default value is <c>infinity</c> or + the value of the <seetype marker="#connect_timeout_client_option"><c>connect_timeout</c></seetype> + option, if present. For connection timeout, use the option <seetype marker="#connect_timeout_client_option"><c>connect_timeout</c></seetype>. </p> diff --git a/lib/ssh/src/ssh.erl b/lib/ssh/src/ssh.erl index bf354c7ef6..f655f0db45 100644 --- a/lib/ssh/src/ssh.erl +++ b/lib/ssh/src/ssh.erl @@ -1,7 +1,7 @@ % %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2004-2020. All Rights Reserved. +%% Copyright Ericsson AB 2004-2021. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -126,25 +126,36 @@ stop() -> %%-------------------------------------------------------------------- %% Description: Starts an ssh connection. %%-------------------------------------------------------------------- --spec connect(OpenTcpSocket, Options) -> {ok,connection_ref()} | {error,term()} when +-define(IS_VALID_OPTIONS(Options), is_list(Options)). +-define(IS_VALID_PORT(Port), (is_integer(Port) andalso Port > 0)). +-define(IS_VALID_TIMEOUT(Timeout), + (Timeout == infinity + orelse (is_integer(Timeout) + andalso Timeout >= 0))). + +-spec connect(OpenTcpSocket, Options) + -> {ok, connection_ref()} + | {error, term()} when OpenTcpSocket :: open_socket(), Options :: client_options(). -connect(OpenTcpSocket, Options) when is_list(Options) -> - connect(OpenTcpSocket, Options, infinity). - +connect(OpenTcpSocket, Options) when ?IS_VALID_OPTIONS(Options) -> + connect(OpenTcpSocket, Options, infinity); +connect(_OpenTcpSocket, Options) -> + bad_arg([{options, Options}]). -spec connect(open_socket(), client_options(), timeout()) -> - {ok,connection_ref()} | {error,term()} + {ok, connection_ref()} | {error, term()} ; (host(), inet:port_number(), client_options()) -> - {ok,connection_ref()} | {error,term()}. - -connect(Host, Port, Options) when is_integer(Port), - Port>0, - is_list(Options) -> - connect(Host, Port, Options, infinity); - -connect(Socket, UserOptions, NegotiationTimeout) when is_list(UserOptions) -> + {ok, connection_ref()} | {error, term()}. + +connect(Host, Port, Options) when ?IS_VALID_PORT(Port), + ?IS_VALID_OPTIONS(Options) -> + Timeout = proplists:get_value(connect_timeout, Options, infinity), + connect(Host, Port, Options, Timeout); +connect(Socket, UserOptions, NegotiationTimeout) + when ?IS_VALID_OPTIONS(UserOptions), + ?IS_VALID_TIMEOUT(NegotiationTimeout) -> case ssh_options:handle_options(client, UserOptions) of {error, Error} -> {error, Error}; @@ -156,26 +167,30 @@ connect(Socket, UserOptions, NegotiationTimeout) when is_list(UserOptions) -> {error,SockError} -> {error,SockError} end - end. - + end; +connect(_HostOrSocket, PortOrOptions, OptionsOrTimeout) -> + bad_arg(PortOrOptions, OptionsOrTimeout). --spec connect(Host, Port, Options, NegotiationTimeout) -> {ok,connection_ref()} | {error,term()} when +-spec connect(Host, Port, Options, NegotiationTimeout) + -> {ok, connection_ref()} + | {error, term()} when Host :: host(), Port :: inet:port_number(), Options :: client_options(), NegotiationTimeout :: timeout(). -connect(Host0, Port, UserOptions, NegotiationTimeout) when is_integer(Port), - Port>0, - is_list(UserOptions) -> +connect(Host0, Port, UserOptions, NegotiationTimeout) + when ?IS_VALID_PORT(Port), + ?IS_VALID_OPTIONS(UserOptions), + ?IS_VALID_TIMEOUT(NegotiationTimeout) -> case ssh_options:handle_options(client, UserOptions) of {error, Reason} -> {error, Reason}; - + Options -> SocketOpts = [{active,false} | ?GET_OPT(socket_options,Options)], Host = mangle_connect_address(Host0, Options), - try + try transport_connect(Host, Port, SocketOpts, Options) of {ok, Socket} -> @@ -188,8 +203,42 @@ connect(Host0, Port, UserOptions, NegotiationTimeout) when is_integer(Port), error:Error -> {error,Error}; Class:Error -> {error, {Class,Error}} end + end; +connect(_Host, Port, UserOptions, NegotiationTimeout) -> + bad_arg([{port, Port}, + {options, UserOptions}, + {timeout, NegotiationTimeout}]). + +bad_arg(Args) -> + hd(bad_args(Args)). + +%% Special handling for finding the incorrect args for connect/3, +%% which has two distinctly different signatures. +bad_arg(Arg2, Arg3) -> + E0 = bad_args([{port, Arg2}, {options, Arg3}]), + E1 = bad_args([{options, Arg2}, {timeout, Arg3}]), + %% Select the case with only one error + case {E0, E1} of + {[Error], _} -> Error; + {_, [Error]} -> Error; + {[Error, _], _} -> Error end. +%% Return list of errors +-spec bad_args([{'options' | 'port' | 'timeout', any()}]) -> + [{'error', term()}]. +bad_args(Args) -> + IsErr = fun(true, _) -> false; + (false, Error) -> {true, {error, Error}} + end, + Check = + fun({options, Arg}) -> IsErr(?IS_VALID_OPTIONS(Arg), invalid_options); + ({timeout, Arg}) -> IsErr(?IS_VALID_TIMEOUT(Arg), invalid_timeout); + ({port, Arg}) -> IsErr(?IS_VALID_PORT(Arg), invalid_port) + end, + + lists:filtermap(Check, Args). + %%%---------------- continue_connect(Socket, Options0, NegTimeout) -> {ok, {SockHost,SockPort}} = inet:sockname(Socket), diff --git a/lib/ssh/src/ssh_acceptor.erl b/lib/ssh/src/ssh_acceptor.erl index 70ebffbea3..b317c79125 100644 --- a/lib/ssh/src/ssh_acceptor.erl +++ b/lib/ssh/src/ssh_acceptor.erl @@ -130,18 +130,27 @@ request_ownership(LSock, SockOwner) -> %%%---------------------------------------------------------------- acceptor_loop(Port, Address, Opts, ListenSocket, AcceptTimeout, SystemSup) -> - case accept(ListenSocket, AcceptTimeout, Opts) of - {ok,Socket} -> - {ok, {FromIP,FromPort}} = inet:peername(Socket), % Just in case of error in next line: - case handle_connection(SystemSup, Address, Port, Opts, Socket) of - {error,Error} -> - catch close(Socket, Opts), - handle_error(Error, Address, Port, FromIP, FromPort); - _ -> - ok - end; - {error,Error} -> - handle_error(Error, Address, Port) + try + case accept(ListenSocket, AcceptTimeout, Opts) of + {ok,Socket} -> + case inet:peername(Socket) of + {ok, {FromIP,FromPort}} -> + case handle_connection(SystemSup, Address, Port, Opts, Socket) of + {error,Error} -> + catch close(Socket, Opts), + handle_error(Error, Address, Port, FromIP, FromPort); + _ -> + ok + end; + {error,Error} -> + handle_error(Error, Address, Port) + end; + {error,Error} -> + handle_error(Error, Address, Port) + end + catch + Class:Err:Stack -> + handle_error({error, {unhandled,Class,Err,Stack}}, Address, Port) end, ?MODULE:acceptor_loop(Port, Address, Opts, ListenSocket, AcceptTimeout, SystemSup). diff --git a/lib/ssh/src/ssh_sftp.erl b/lib/ssh/src/ssh_sftp.erl index 22997d863a..015b2f2a30 100644 --- a/lib/ssh/src/ssh_sftp.erl +++ b/lib/ssh/src/ssh_sftp.erl @@ -135,7 +135,8 @@ start_channel(Dest) -> ) -> {ok,pid(),ssh:connection_ref()} | {error,reason()} . -start_channel(Cm, UserOptions) when is_pid(Cm) -> +start_channel(Cm, UserOptions0) when is_pid(Cm) -> + UserOptions = legacy_timeout(UserOptions0), Timeout = proplists:get_value(timeout, UserOptions, infinity), {_SshOpts, ChanOpts, SftpOpts} = handle_options(UserOptions), WindowSize = proplists:get_value(window_size, ChanOpts, ?XFER_WINDOW_SIZE), @@ -158,7 +159,8 @@ start_channel(Cm, UserOptions) when is_pid(Cm) -> Error end; -start_channel(Dest, UserOptions) -> +start_channel(Dest, UserOptions0) -> + UserOptions = legacy_timeout(UserOptions0), {SshOpts, ChanOpts, SftpOpts} = handle_options(UserOptions), case ssh:is_host(Dest, SshOpts) of true -> @@ -167,9 +169,7 @@ start_channel(Dest, UserOptions) -> false -> %% No, it is probably not a Host, must be a socket Socket = Dest, - Timeout = % A mixture of ssh:connect and ssh_sftp:start_channel: - proplists:get_value(connect_timeout, SshOpts, - proplists:get_value(timeout, SftpOpts, infinity)), + Timeout = proplists:get_value(connect_timeout, SshOpts, infinity), case ssh:connect(Socket, SshOpts, Timeout) of {ok,Cm} -> case start_channel(Cm, ChanOpts ++ SftpOpts) of @@ -193,15 +193,10 @@ start_channel(Dest, UserOptions) -> ) -> {ok,pid(),ssh:connection_ref()} | {error,reason()}. -start_channel(Host, Port, UserOptions) -> +start_channel(Host, Port, UserOptions0) -> + UserOptions = legacy_timeout(UserOptions0), + Timeout = proplists:get_value(connect_timeout, UserOptions, infinity), {SshOpts, _ChanOpts, _SftpOpts} = handle_options(UserOptions), - Timeout = % A mixture of ssh:connect and ssh_sftp:start_channel: - case proplists:get_value(connect_timeout, UserOptions) of - undefined -> - proplists:get_value(timeout, UserOptions, infinity); - TO -> - TO - end, case ssh:connect(Host, Port, SshOpts, Timeout) of {ok, Cm} -> case start_channel(Cm, UserOptions) of @@ -1228,6 +1223,21 @@ terminate(_Reason, State) -> %%==================================================================== %% Internal functions %%==================================================================== +legacy_timeout(UserOptions) -> + %% Make both connect_timeout and timeout defined if exaclty one of them is defined: + case {proplists:get_value(connect_timeout, UserOptions), + proplists:get_value(timeout, UserOptions)} of + {undefined, undefined} -> + UserOptions; + {undefined, TO} -> + [{connect_timeout,TO} | UserOptions]; + {TO, undefined} -> + [{timeout,TO} | UserOptions]; + {_, _} -> + UserOptions + end. + + handle_options(UserOptions) -> handle_options(UserOptions, [], [], []). diff --git a/lib/ssh/src/ssh_transport.erl b/lib/ssh/src/ssh_transport.erl index 64998631db..0b91e0258c 100644 --- a/lib/ssh/src/ssh_transport.erl +++ b/lib/ssh/src/ssh_transport.erl @@ -88,33 +88,45 @@ clear_default_algorithms_env() -> | no_return() % error(Reason) . default_algorithms() -> + FipsMode = crypto:info_fips(), case application:get_env(ssh, ?DEFAULT_ALGS) of undefined -> - %% Not cached, have to build the default, connection independent - %% set of algorithms: - Opts = get_alg_conf(), - Algs1 = - case proplists:get_value(preferred_algorithms, Opts) of - undefined -> - [{K,default_algorithms1(K)} || K <- algo_classes()]; - Algs0 -> - {true,Algs01} = ssh_options:check_preferred_algorithms(Algs0), - Algs01 - end, - Algs = - case proplists:get_value(modify_algorithms, Opts) of - undefined -> - Algs1; - Modifications -> - ssh_options:initial_default_algorithms(Algs1, Modifications) - end, - application:set_env(ssh, ?DEFAULT_ALGS, Algs), + Algs = build_cache(), + application:set_env(ssh, ?DEFAULT_ALGS, {FipsMode,Algs}), Algs; - {ok,Algs} -> + {ok,{FipsMode,Algs}} -> + %% Cached, and the FIPS mode is the same now as when it was cached. + Algs; + + {ok,{_OtherFipsMode,_Algs}} -> + %% Cached, but the FIPS mode has changed. + Algs = build_cache(), + application:set_env(ssh, ?DEFAULT_ALGS, {FipsMode,Algs}), Algs end. +build_cache() -> + Opts = get_alg_conf(), + Algs1 = + case proplists:get_value(preferred_algorithms, Opts) of + undefined -> + [{K,default_algorithms1(K)} || K <- algo_classes()]; + Algs0 -> + {true,Algs01} = ssh_options:check_preferred_algorithms(Algs0), + Algs01 + end, + Algs = + case proplists:get_value(modify_algorithms, Opts) of + undefined -> + Algs1; + Modifications -> + ssh_options:initial_default_algorithms(Algs1, Modifications) + end, + Algs. + + + get_alg_conf() -> [{T,L} || T <- [preferred_algorithms, modify_algorithms], L <- [application:get_env(ssh, T, [])], @@ -139,10 +151,17 @@ algo_two_spec_class(_) -> false. default_algorithms(Tag) -> + FipsMode = crypto:info_fips(), case application:get_env(ssh, ?DEFAULT_ALGS) of undefined -> default_algorithms1(Tag); - {ok,Algs} -> + {ok,{FipsMode,Algs}} -> + %% Cached, and the FIPS mode is the same now as when it was cached. + proplists:get_value(Tag, Algs, []); + {ok,{_OtherFipsMode,_Algs}} -> + %% Cached, but the FIPS mode has changed. + Algs = build_cache(), + application:set_env(ssh, ?DEFAULT_ALGS, {FipsMode,Algs}), proplists:get_value(Tag, Algs, []) end. @@ -356,7 +375,8 @@ s2c(Key, Algs) -> x2y(server2client, Key, Algs). x2y(DirectionKey, Key, Algs) -> to_strings(proplists:get_value(DirectionKey, get_algs(Key,Algs))). -get_algs(Key, Algs) -> proplists:get_value(Key, Algs, default_algorithms(Key)). +get_algs(Key, {_FipsMode,Algs}) when is_list(Algs) -> proplists:get_value(Key, Algs, default_algorithms(Key)); +get_algs(Key, Algs) when is_list(Algs) -> proplists:get_value(Key, Algs, default_algorithms(Key)). to_strings(L) -> lists:map(fun erlang:atom_to_list/1, L). diff --git a/lib/ssh/test/ssh_algorithms_SUITE.erl b/lib/ssh/test/ssh_algorithms_SUITE.erl index 1b8c09c088..7a7a236a6c 100644 --- a/lib/ssh/test/ssh_algorithms_SUITE.erl +++ b/lib/ssh/test/ssh_algorithms_SUITE.erl @@ -86,6 +86,24 @@ groups() -> || {Tag,Algs} <- ErlAlgos ++ DoubleAlgos, Alg <- Algs], + ct:log( + "ErlAlgos = ~p~n" + "SshcAlgos = ~p~n" + "SshdAlgos = ~p~n" + "DoubleAlgos = ~p~n" + "TypeSSH = ~p~n" + "TagGroupSet = ~p~n" + "AlgoTcSet = ~p~n" + ,[ + ErlAlgos, + SshcAlgos, + SshdAlgos, + DoubleAlgos, + TypeSSH, + TagGroupSet, + AlgoTcSet + ]), + TagGroupSet ++ AlgoTcSet. tags() -> [kex,cipher,mac,compression,public_key]. @@ -146,50 +164,76 @@ init_per_group(Group, Config) -> Tag = proplists:get_value(name, hd(proplists:get_value(tc_group_path, Config))), Alg = Group, - init_per_group(Tag, Alg, Config) + Algs = split(Tag, Alg), + SupportedAlgs = proplists:get_value(Tag, ssh_transport:supported_algorithms()), + PA = + case Algs of + [_] -> + [Alg]; + [A1,A2] when Tag == public_key -> + [A1,A2]; + [A1,A2] -> + [{client2server,[A1]}, + {server2client,[A2]}] + end, + case lists:foldl(fun({K,As}, Acc) -> + ct:log("~p:~p K=~p, As=~p, SupportedAlgs=~p", [?MODULE,?LINE,K,As,SupportedAlgs]), + SAs = proplists:get_value(K,SupportedAlgs), + lists:foldl(fun(A1, Acc1) -> + case lists:member(A1, SAs) of + true -> Acc1; + false -> [A1|Acc1] + end + end, Acc, As); + (A, Acc) when is_atom(hd(SupportedAlgs)) -> + ct:log("~p:~p A=~p, SupportedAlgs=~p", [?MODULE,?LINE,A,SupportedAlgs]), + case lists:member(A, SupportedAlgs) of + true -> Acc; + false -> [A|Acc] + end; + (A, Acc) when is_tuple(hd(SupportedAlgs)) -> + ct:log("~p:~p A=~p, SupportedAlgs=~p", [?MODULE,?LINE,A,SupportedAlgs]), + [{_,S1},{_,S2}] = SupportedAlgs, + + case lists:member(A, S1) andalso + lists:member(A, S2) of + true -> Acc; + false -> [A|Acc] + end + end, [], PA) of + [] -> + init_per_group(Tag, Algs, Alg, PA, Config); + L -> + ct:log("~p:~p Tag ~p, Alg ~p, Algs ~p, PA ~p,~nSupportedAlgs ~p", [?MODULE,?LINE, Tag, Alg, Algs, PA, SupportedAlgs]), + {skip,io_lib:format("Unsupported ~p: ~p", [Tag,L])} + end end. -init_per_group(public_key=Tag, Alg, Config) -> - PA = - case split(Tag, Alg) of - [_] -> - [Alg]; - [A1,A2] -> - [A1,A2] - end, +init_per_group(Tag, Algs, Alg, PA, Config) -> OtherAlgs = [{T,L} || {T,L} <- ssh_transport:supported_algorithms(), T=/=Tag], - ct:log("Init tests for public_key ~p~nOtherAlgs=~p",[PA,OtherAlgs]), + ct:log("init_per_group Tag ~p, Alg ~p, Algs ~p ,PA ~p,~nOtherAlgs ~p", [Tag, Alg, Algs, PA, OtherAlgs]), PrefAlgs = {preferred_algorithms,[{Tag,PA}|OtherAlgs]}, - %% Daemon started later in init_per_testcase - try - setup_pubkey(PA, - [{pref_algs,PrefAlgs}, - {tag_alg,{Tag,PA}} - | Config]) - catch - _C:_E:_S -> - ct:log("Exception ~p:~p~n~p",[_C,_E,_S]), - {skip, io_lib:format("Unsupported: ~p",[Alg])} - end; - -init_per_group(Tag, Alg, Config) -> - PA = - case split(Tag, Alg) of - [_] -> - [Alg]; - [A1,A2] -> - [{client2server,[A1]}, - {server2client,[A2]}] - end, - OtherAlgs = [{T,L} || {T,L} <- ssh_transport:supported_algorithms(), T=/=Tag], - ct:log("Init tests for tag=~p alg=~p~nOtherAlgs=~p",[Tag,PA,OtherAlgs]), - PrefAlgs = {preferred_algorithms,[{Tag,PA}|OtherAlgs]}, - start_std_daemon([PrefAlgs], - [{pref_algs,PrefAlgs}, - {tag_alg,{Tag,[Alg]}} - | Config]). + case Tag of + public_key -> + %% Daemon started later in init_per_testcase + try + setup_pubkey(PA, + [{pref_algs,PrefAlgs}, + {tag_alg,{Tag,PA}} + | Config]) + catch + _C:_E:_S -> + ct:log("Exception ~p:~p~n~p",[_C,_E,_S]), + {skip, io_lib:format("Unsupported: ~p",[Alg])} + end; + _ -> + start_std_daemon([PrefAlgs], + [{pref_algs,PrefAlgs}, + {tag_alg,{Tag,[Alg]}} + | Config]) + end. end_per_group(_Alg, Config) -> case proplists:get_value(srvr_pid,Config) of diff --git a/lib/ssh/test/ssh_compat_SUITE.erl b/lib/ssh/test/ssh_compat_SUITE.erl index 44725705e0..262ea0d4ca 100644 --- a/lib/ssh/test/ssh_compat_SUITE.erl +++ b/lib/ssh/test/ssh_compat_SUITE.erl @@ -35,7 +35,9 @@ init_per_suite/1, end_per_suite/1, init_per_group/2, - end_per_group/2 + end_per_group/2, + init_per_testcase/2, + end_per_testcase/2 ]). -export([ @@ -193,6 +195,26 @@ end_per_group(G, Config) -> ok end. + +init_per_testcase(TC, Config) when TC==login_otp_is_client ; + TC==all_algorithms_sftp_exec_reneg_otp_is_client -> + case proplists:get_value(ssh_version, Config) of + "openssh4.4p1-openssl0.9.8c" -> {skip, "Not tested"}; + "openssh4.5p1-openssl0.9.8m" -> {skip, "Not tested"}; + "openssh5.0p1-openssl0.9.8za" -> {skip, "Not tested"}; + "openssh6.2p2-openssl0.9.8c" -> {skip, "Not tested"}; + "openssh6.3p1-openssl0.9.8zh" -> {skip, "Not tested"}; + "openssh6.6p1-openssl1.0.2n" -> {skip, "Not tested"}; + _ -> + Config + end; +init_per_testcase(_, Config) -> + Config. + + +end_per_testcase(_TC, _Config) -> + ok. + %%-------------------------------------------------------------------- %% Test Cases -------------------------------------------------------- %%-------------------------------------------------------------------- diff --git a/lib/ssh/test/ssh_connection_SUITE.erl b/lib/ssh/test/ssh_connection_SUITE.erl index dbc0c45fff..1e79f0a622 100644 --- a/lib/ssh/test/ssh_connection_SUITE.erl +++ b/lib/ssh/test/ssh_connection_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2008-2020. All Rights Reserved. +%% Copyright Ericsson AB 2008-2021. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -43,6 +43,21 @@ big_cat/1, connect_sock_not_passive/1, connect_sock_not_tcp/1, + connect2_invalid_options/1, + connect_invalid_port/1, + connect_invalid_timeout_0/1, + connect_invalid_timeout_1/1, + connect_invalid_options/1, + connect3_invalid_port/1, + connect3_invalid_options/1, + connect3_invalid_timeout_0/1, + connect3_invalid_timeout_1/1, + connect3_invalid_both/1, + connect4_invalid_two_0/1, + connect4_invalid_two_1/1, + connect4_invalid_two_2/1, + connect4_invalid_three/1, + connect_timeout/1, daemon_sock_not_passive/1, daemon_sock_not_tcp/1, do_interrupted_send/3, @@ -132,6 +147,21 @@ all() -> start_shell_sock_daemon_exec_multi, encode_decode_pty_opts, connect_sock_not_tcp, + connect2_invalid_options, + connect_invalid_port, + connect_invalid_options, + connect_invalid_timeout_0, + connect_invalid_timeout_1, + connect3_invalid_port, + connect3_invalid_options, + connect3_invalid_timeout_0, + connect3_invalid_timeout_1, + connect3_invalid_both, + connect4_invalid_two_0, + connect4_invalid_two_1, + connect4_invalid_two_2, + connect4_invalid_three, + connect_timeout, daemon_sock_not_tcp, gracefull_invalid_version, gracefull_invalid_start, @@ -240,6 +270,139 @@ simple_exec_two_socks(_Config) -> end. %%-------------------------------------------------------------------- +connect2_invalid_options(_Config) -> + {error, invalid_options} = ssh:connect(bogus_socket, {bad, option}). + +connect3_invalid_port(_Config) -> + {error, invalid_port} = ssh:connect(bogus_host, noport, [{key, value}]). + +connect3_invalid_options(_Config) -> + {error, invalid_options} = ssh:connect(bogus_host, 1337, bad_options). + +connect3_invalid_timeout_0(_Config) -> + {error, invalid_timeout} = + ssh:connect(bogus_socket, [{key, value}], short). + +connect3_invalid_timeout_1(_Config) -> + {error, invalid_timeout} = + ssh:connect(bogus_socket, [{key, value}], -1). + +connect3_invalid_both(_Config) -> + %% The actual reason is implementation dependent. + {error, _Reason} = + ssh:connect(bogus, no_list_or_port, no_list_or_timeout). + +connect_invalid_port(Config) -> + {Pid, Host, _Port, UserDir} = daemon_start(Config), + + {error, invalid_port} = + ssh:connect(Host, undefined, + [{silently_accept_hosts, true}, + {user, "foo"}, + {password, "morot"}, + {user_interaction, false}, + {user_dir, UserDir}], + infinity), + + ssh:stop_daemon(Pid). + +connect_invalid_timeout_0(Config) -> + {Pid, Host, Port, UserDir} = daemon_start(Config), + + {error, invalid_timeout} = + ssh:connect(Host, Port, + [{silently_accept_hosts, true}, + {user, "foo"}, + {password, "morot"}, + {user_interaction, false}, + {user_dir, UserDir}], + longer), + + ssh:stop_daemon(Pid). + +connect_invalid_timeout_1(Config) -> + {Pid, Host, Port, UserDir} = daemon_start(Config), + + {error, invalid_timeout} = + ssh:connect(Host, Port, + [{silently_accept_hosts, true}, + {user, "foo"}, + {password, "morot"}, + {user_interaction, false}, + {user_dir, UserDir}], + -1), + + ssh:stop_daemon(Pid). + +connect_invalid_options(Config) -> + {Pid, Host, Port, _UserDir} = daemon_start(Config), + + {error, invalid_options} = + ssh:connect(Host, Port, + {user, "foo"}, + infinity), + + ssh:stop_daemon(Pid). + +%% Two out three arguments incorrect. Three possibilities. +connect4_invalid_two_0(Config) -> + {Pid, Host, _Port, _UserDir} = daemon_start(Config), + + %% Actual error implementation dependent + {error, _} = + ssh:connect(Host, noport, + {user, "foo"}, + infinity), + + ssh:stop_daemon(Pid). + +connect4_invalid_two_1(Config) -> + {Pid, Host, Port, _UserDir} = daemon_start(Config), + + %% Actual error implementation dependent + {error, _} = + ssh:connect(Host, Port, + {user, "foo"}, + short), + + ssh:stop_daemon(Pid). + +connect4_invalid_two_2(Config) -> + {Pid, Host, Port, _UserDir} = daemon_start(Config), + + %% Actual error implementation dependent + {error, _} = + ssh:connect(Host, newport, + [{user, "foo"}], + -1), + + ssh:stop_daemon(Pid). + +%% All three args incorrect +connect4_invalid_three(Config) -> + {Pid, Host, Port, _UserDir} = daemon_start(Config), + + %% Actual error implementation dependent + {error, _} = + ssh:connect(Host, teleport, + {user, "foo"}, + fortnight), + + ssh:stop_daemon(Pid). + +daemon_start(Config) -> + PrivDir = proplists:get_value(priv_dir, Config), + UserDir = filename:join(PrivDir, nopubkey), % to make sure we don't use public-key-auth + file:make_dir(UserDir), + SysDir = proplists:get_value(data_dir, Config), + + {Pid, Host, Port} = ssh_test_lib:daemon([{system_dir, SysDir}, + {user_dir, UserDir}, + {password, "morot"}, + {exec, fun ssh_exec_echo/1}]), + {Pid, Host, Port, UserDir}. + +%%-------------------------------------------------------------------- connect_sock_not_tcp(_Config) -> {ok,Sock} = gen_udp:open(0, []), {error, not_tcp_socket} = ssh:connect(Sock, [{save_accepted_host, false}, @@ -248,6 +411,15 @@ connect_sock_not_tcp(_Config) -> gen_udp:close(Sock). %%-------------------------------------------------------------------- +connect_timeout(_Config) -> + {ok,Sl} = gen_tcp:listen(0, []), + {ok, {_,Port}} = inet:sockname(Sl), + {error,timeout} = ssh:connect(loopback, Port, [{connect_timeout,2000}, + {save_accepted_host, false}, + {silently_accept_hosts, true}]), + gen_tcp:close(Sl). + +%%-------------------------------------------------------------------- daemon_sock_not_tcp(_Config) -> {ok,Sock} = gen_udp:open(0, []), {error, not_tcp_socket} = ssh:daemon(Sock), @@ -1239,7 +1411,8 @@ kex_error(Config) -> stop_listener(Config) when is_list(Config) -> PrivDir = proplists:get_value(priv_dir, Config), - UserDir = filename:join(PrivDir, nopubkey), % to make sure we don't use public-key-auth + %% to make sure we don't use public-key-auth + UserDir = filename:join(PrivDir, nopubkey), file:make_dir(UserDir), SysDir = proplists:get_value(data_dir, Config), @@ -1248,13 +1421,14 @@ stop_listener(Config) when is_list(Config) -> {password, "morot"}, {exec, fun ssh_exec_echo/1}]), - ConnectionRef0 = ssh_test_lib:connect(Host, Port, [{silently_accept_hosts, true}, - {user, "foo"}, - {password, "morot"}, - {user_interaction, false}, - {user_dir, UserDir}]), - - {ok, ChannelId0} = ssh_connection:session_channel(ConnectionRef0, infinity), + ConnectionRef0 = ssh_test_lib:connect(Host, Port, + [{silently_accept_hosts, true}, + {user, "foo"}, + {password, "morot"}, + {user_interaction, false}, + {user_dir, UserDir}]), + {ok, ChannelId0} = + ssh_connection:session_channel(ConnectionRef0, infinity), ssh:stop_listener(Host, Port), @@ -1268,7 +1442,8 @@ stop_listener(Config) when is_list(Config) -> success = ssh_connection:exec(ConnectionRef0, ChannelId0, "testing", infinity), receive - {ssh_cm, ConnectionRef0, {data, ChannelId0, 0, <<"echo testing\n">>}} -> + {ssh_cm, ConnectionRef0, + {data, ChannelId0, 0, <<"echo testing\n">>}} -> ok after 5000 -> ct:fail("Exec Timeout") @@ -1279,11 +1454,12 @@ stop_listener(Config) when is_list(Config) -> {password, "potatis"}, {exec, fun ssh_exec_echo/1}]) of {Pid1, Host, Port} -> - ConnectionRef1 = ssh_test_lib:connect(Host, Port, [{silently_accept_hosts, true}, - {user, "foo"}, - {password, "potatis"}, - {user_interaction, true}, - {user_dir, UserDir}]), + ConnectionRef1 = + ssh_test_lib:connect(Host, Port, [{silently_accept_hosts, true}, + {user, "foo"}, + {password, "potatis"}, + {user_interaction, true}, + {user_dir, UserDir}]), {error, _} = ssh:connect(Host, Port, [{silently_accept_hosts, true}, {save_accepted_host, false}, {user, "foo"}, diff --git a/lib/ssh/vsn.mk b/lib/ssh/vsn.mk index f26eb9b006..4812d27d10 100644 --- a/lib/ssh/vsn.mk +++ b/lib/ssh/vsn.mk @@ -1,4 +1,4 @@ #-*-makefile-*- ; force emacs to enter makefile-mode -SSH_VSN = 4.12.4 +SSH_VSN = 4.12.5 APP_VSN = "ssh-$(SSH_VSN)" diff --git a/lib/ssl/doc/src/notes.xml b/lib/ssl/doc/src/notes.xml index 7ca6cc2e5b..b8cab8cce7 100644 --- a/lib/ssl/doc/src/notes.xml +++ b/lib/ssl/doc/src/notes.xml @@ -27,6 +27,53 @@ </header> <p>This document describes the changes made to the SSL application.</p> +<section><title>SSL 10.5.3</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Correct typo of ECC curve name in signature algorithm + handling. Will make the signature algorithm + ecdsa_secp521r1_sha512 succeed.</p> + <p> + Own Id: OTP-17756 Aux Id: GH-5383, PR-5397 </p> + </item> + <item> + <p> + Suppress authenticity warning when option verify_none is + explicitly supplied.</p> + <p> + Own Id: OTP-17757 Aux Id: GH-5352, PR-5395 </p> + </item> + </list> + </section> + +</section> + +<section><title>SSL 10.5.2</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Fix TLS-1.2 RSA-PSS negotiation and also fix broken + certificate request message for pre-TLS-1.3 servers.</p> + <p> + Own Id: OTP-17688 Aux Id: GH-5255 </p> + </item> + <item> + <p> + Fix CRL issuer verification that under some circumstances + could fail with a function_clause error.</p> + <p> + Own Id: OTP-17723 Aux Id: GH-5300 </p> + </item> + </list> + </section> + +</section> + <section><title>SSL 10.5.1</title> <section><title>Fixed Bugs and Malfunctions</title> diff --git a/lib/ssl/doc/src/ssl.xml b/lib/ssl/doc/src/ssl.xml index 700d8be5fb..0b4bf3b3c7 100644 --- a/lib/ssl/doc/src/ssl.xml +++ b/lib/ssl/doc/src/ssl.xml @@ -485,7 +485,7 @@ version. </p> <note> <p> TLS-1.2 algorithms will not be negotiated for TLS-1.3, but TLS-1.3 RSASSA-PSS <seetype marker="#rsassa_pss_scheme">rsassa_pss_scheme()</seetype> - signature schemes may be negotiated also for TLS-1.2 from @OTP-16590@ + signature schemes may be negotiated also for TLS-1.2 from 24.1 (fully working from 24.1.3). However if TLS-1.3 is negotiated when both TLS-1.3 and TLS-1.2 is supported using defaults, the corresponding TLS-1.2 algorithms to the TLS-1.3 legacy signature schemes will be considered as the legacy schemes and applied only to certificate signatures. </p> @@ -519,7 +519,7 @@ version. <p> The client will send a <c>signature_algorithms_cert</c> extension (in the client hello message), if TLS version 1.2 - (back-ported to TLS 1.2 in @OTP-16590@) or later is used, and + (back-ported to TLS 1.2 in 24.1) or later is used, and the signature_algs_cert option is explicitly specified. By default, only the <seetype marker="#signature_algs">signature_algs</seetype> extension @@ -884,7 +884,7 @@ fun(srp, Username :: binary(), UserState :: term()) -> <datatype> <name name="ssl_imp"/> - <desc><p>Deprecated since OTP-17, has no affect.</p></desc> + <desc><p>Deprecated since OTP-17, has no effect.</p></desc> </datatype> <datatype> diff --git a/lib/ssl/src/Makefile b/lib/ssl/src/Makefile index 9cbee91946..1ba0f49f91 100644 --- a/lib/ssl/src/Makefile +++ b/lib/ssl/src/Makefile @@ -103,6 +103,7 @@ MODULES= \ tls_client_ticket_store \ tls_dist_sup \ tls_dist_server_sup \ + tls_dyn_connection_sup\ tls_sender \ tls_server_session_ticket\ tls_server_session_ticket_sup\ diff --git a/lib/ssl/src/dtls_connection.erl b/lib/ssl/src/dtls_connection.erl index 1a4f001273..47e0e9f90a 100644 --- a/lib/ssl/src/dtls_connection.erl +++ b/lib/ssl/src/dtls_connection.erl @@ -132,6 +132,8 @@ -export([renegotiate/2]). +-export([alert_or_reset_connection/3]). %% Code re-use from dtls_gen_connection. + %% gen_statem state functions -export([initial_hello/3, config_error/3, @@ -200,14 +202,19 @@ initial_hello({call, From}, {start, Timeout}, #state{static_env = #static_env{host = Host, port = Port, role = client, + socket = {_, Socket}, + transport_cb = Transport, session_cache = Cache, session_cache_cb = CacheCb}, + protocol_specific = PS, handshake_env = #handshake_env{renegotiation = {Renegotiation, _}}, connection_env = CEnv, ssl_options = #{versions := Versions} = SslOpts, session = #session{own_certificates = OwnCerts} = NewSession, connection_states = ConnectionStates0 } = State0) -> + Packages = maps:get(active_n, PS), + dtls_socket:setopts(Transport, Socket, [{active,Packages}]), Session = ssl_session:client_select_session({Host, Port, SslOpts}, Cache, CacheCb, NewSession), Hello = dtls_handshake:client_hello(Host, Port, ConnectionStates0, SslOpts, Session#session.session_id, Renegotiation, OwnCerts), @@ -223,17 +230,15 @@ initial_hello({call, From}, {start, Timeout}, State1#state{connection_env = CEnv#connection_env{negotiated_version = HelloVersion}}), State = State2#state{connection_env = CEnv#connection_env{negotiated_version = Version}, %% RequestedVersion - session = Session, - start_or_recv_from = From}, + session = Session, + start_or_recv_from = From, + protocol_specific = PS#{active_n_toggle := false} + }, dtls_gen_connection:next_event(hello, no_record, State, [{{timeout, handshake}, Timeout, close} | Actions]); initial_hello({call, _} = Type, Event, #state{static_env = #static_env{role = server}, - protocol_specific = PS} = State) -> - Result = ssl_gen_statem:?FUNCTION_NAME(Type, Event, - State#state{protocol_specific = - PS#{current_cookie_secret => dtls_v1:cookie_secret(), - previous_cookie_secret => <<>>, - ignored_alerts => 0, - max_ignored_alerts => 10}}), + protocol_specific = PS0} = State) -> + PS = PS0#{current_cookie_secret => dtls_v1:cookie_secret(), previous_cookie_secret => <<>>}, + Result = ssl_gen_statem:?FUNCTION_NAME(Type, Event, State#state{protocol_specific = PS}), erlang:send_after(dtls_v1:cookie_timeout(), self(), new_cookie_secret), Result; initial_hello(Type, Event, State) -> @@ -268,7 +273,7 @@ hello(internal, #client_hello{cookie = <<>>, handshake_env = HsEnv, connection_env = CEnv, protocol_specific = #{current_cookie_secret := Secret}} = State0) -> - case tls_dtls_connection:handle_sni_extension(State0, Hello) of + try tls_dtls_connection:handle_sni_extension(State0, Hello) of #state{} = State1 -> {ok, {IP, Port}} = dtls_socket:peername(Transport, Socket), Cookie = dtls_handshake:cookie(Secret, IP, Port, Hello), @@ -285,9 +290,9 @@ hello(internal, #client_hello{cookie = <<>>, State#state{handshake_env = HsEnv#handshake_env{ tls_handshake_history = ssl_handshake:init_handshake_history()}}, - Actions); - #alert{} = Alert -> - ssl_gen_statem:handle_own_alert(Alert, Version,?FUNCTION_NAME, State0) + Actions) + catch throw:#alert{} = Alert -> + alert_or_reset_connection(Alert, ?FUNCTION_NAME, State0) end; hello(internal, #hello_verify_request{cookie = Cookie}, #state{static_env = #static_env{role = client, @@ -317,27 +322,18 @@ hello(internal, #hello_verify_request{cookie = Cookie}, State = State2#state{connection_env = CEnv#connection_env{negotiated_version = Version} % RequestedVersion }, dtls_gen_connection:next_event(?FUNCTION_NAME, no_record, State, Actions); -hello(internal, #client_hello{extensions = Extensions, client_version = ClientVersion} = Hello, +hello(internal, #client_hello{extensions = Extensions} = Hello, #state{ssl_options = #{handshake := hello}, handshake_env = HsEnv, start_or_recv_from = From} = State0) -> - case tls_dtls_connection:handle_sni_extension(State0, Hello) of + try tls_dtls_connection:handle_sni_extension(State0, Hello) of #state{} = State -> {next_state, user_hello, State#state{start_or_recv_from = undefined, handshake_env = HsEnv#handshake_env{hello = Hello}}, - [{reply, From, {ok, Extensions}}]}; - #alert{} = Alert -> - ssl_gen_statem:handle_own_alert(Alert, ClientVersion, ?FUNCTION_NAME, State0) + [{reply, From, {ok, Extensions}}]} + catch throw:#alert{} = Alert -> + alert_or_reset_connection(Alert, ?FUNCTION_NAME, State0) end; -hello(internal, #server_hello{extensions = Extensions} = Hello, - #state{ssl_options = #{ - handshake := hello}, - handshake_env = HsEnv, - start_or_recv_from = From} = State) -> - {next_state, user_hello, State#state{start_or_recv_from = undefined, - handshake_env = HsEnv#handshake_env{ - hello = Hello}}, - [{reply, From, {ok, Extensions}}]}; hello(internal, #client_hello{cookie = Cookie} = Hello, #state{static_env = #static_env{role = server, transport_cb = Transport, socket = Socket}, @@ -350,32 +346,42 @@ hello(internal, #client_hello{cookie = Cookie} = Hello, #state{static_env = #sta handle_client_hello(Hello, State); _ -> case dtls_handshake:cookie(PSecret, IP, Port, Hello) of - Cookie -> + Cookie -> handle_client_hello(Hello, State); _ -> %% Handle bad cookie as new cookie request RFC 6347 4.1.2 hello(internal, Hello#client_hello{cookie = <<>>}, State) end end; +hello(internal, #server_hello{extensions = Extensions} = Hello, + #state{ssl_options = #{ + handshake := hello}, + handshake_env = HsEnv, + start_or_recv_from = From} = State) -> + {next_state, user_hello, State#state{start_or_recv_from = undefined, + handshake_env = HsEnv#handshake_env{ + hello = Hello}}, + [{reply, From, {ok, Extensions}}]}; + hello(internal, #server_hello{} = Hello, #state{ static_env = #static_env{role = client}, handshake_env = #handshake_env{ - renegotiation = {Renegotiation, _}, - ocsp_stapling_state = OcspState0} = HsEnv, - connection_env = #connection_env{negotiated_version = ReqVersion}, + renegotiation = {Renegotiation, _}, + ocsp_stapling_state = OcspState0} = HsEnv, connection_states = ConnectionStates0, session = #session{session_id = OldId}, ssl_options = SslOptions} = State) -> - case dtls_handshake:hello(Hello, SslOptions, ConnectionStates0, Renegotiation, OldId) of - #alert{} = Alert -> - ssl_gen_statem:handle_own_alert(Alert, ReqVersion, ?FUNCTION_NAME, State); - {Version, NewId, ConnectionStates, ProtoExt, Protocol, OcspState} -> - tls_dtls_connection:handle_session(Hello, - Version, NewId, ConnectionStates, ProtoExt, Protocol, - State#state{handshake_env = - HsEnv#handshake_env{ - ocsp_stapling_state = maps:merge(OcspState0,OcspState)}}) + try + {Version, NewId, ConnectionStates, ProtoExt, Protocol, OcspState} = + dtls_handshake:hello(Hello, SslOptions, ConnectionStates0, Renegotiation, OldId), + tls_dtls_connection:handle_session(Hello, + Version, NewId, ConnectionStates, ProtoExt, Protocol, + State#state{handshake_env = + HsEnv#handshake_env{ + ocsp_stapling_state = maps:merge(OcspState0,OcspState)}}) + catch throw:#alert{} = Alert -> + ssl_gen_statem:handle_own_alert(Alert, ?FUNCTION_NAME, State) end; hello(internal, {handshake, {#client_hello{cookie = <<>>} = Handshake, _}}, State) -> %% Initial hello should not be in handshake history @@ -487,12 +493,19 @@ cipher(Type, Event, State) -> gen_handshake(?FUNCTION_NAME, Type, Event, State). %%-------------------------------------------------------------------- --spec connection(gen_statem:event_type(), +-spec connection(gen_statem:event_type(), #hello_request{} | #client_hello{}| term(), #state{}) -> gen_statem:state_function_result(). %%-------------------------------------------------------------------- -connection(enter, _, State) -> - {keep_state, State}; +connection(enter, _, #state{connection_states = Cs0} = State0) -> + State = case maps:is_key(previous_cs, Cs0) of + false -> + State0; + true -> + Cs = maps:remove(previous_cs, Cs0), + State0#state{connection_states = Cs} + end, + {keep_state, State}; connection(info, Event, State) -> gen_info(Event, ?FUNCTION_NAME, State); connection(internal, #hello_request{}, #state{static_env = #static_env{host = Host, @@ -541,14 +554,42 @@ connection(internal, #client_hello{}, #state{static_env = #static_env{role = ser State1 = dtls_gen_connection:send_alert(Alert, State0), {Record, State} = ssl_gen_statem:prepare_connection(State1, Connection), dtls_gen_connection:next_event(?FUNCTION_NAME, Record, State); +connection(internal, new_connection, #state{ssl_options=SSLOptions, + handshake_env=HsEnv, + connection_states = OldCs} = State) -> + case maps:get(previous_cs, OldCs, undefined) of + undefined -> + #{beast_mitigation := BeastMitigation} = SSLOptions, + ConnectionStates0 = dtls_record:init_connection_states(server, BeastMitigation), + ConnectionStates = ConnectionStates0#{previous_cs => OldCs}, + {next_state, hello, State#state{handshake_env = HsEnv#handshake_env{renegotiation = {false, first}}, + connection_states = ConnectionStates}}; + _ -> + %% Someone spamming new_connection, just drop them + {keep_state, State} + end; + connection({call, From}, {application_data, Data}, State) -> try send_application_data(Data, From, ?FUNCTION_NAME, State) catch throw:Error -> ssl_gen_statem:hibernate_after(?FUNCTION_NAME, State, [{reply, From, Error}]) end; +connection({call, From}, {downgrade, Pid}, + #state{connection_env = CEnv, + static_env = #static_env{transport_cb = Transport, + socket = {_Server, Socket} = DTLSSocket}} = State) -> + %% For testing purposes, downgrades without noticing the server + dtls_socket:setopts(Transport, Socket, [{active, false}, {packet, 0}, {mode, binary}]), + Transport:controlling_process(Socket, Pid), + {stop_and_reply, {shutdown, normal}, {reply, From, {ok, DTLSSocket}}, + State#state{connection_env = CEnv#connection_env{socket_terminated = true}}}; connection(Type, Event, State) -> - tls_dtls_connection:?FUNCTION_NAME(Type, Event, State). + try + tls_dtls_connection:?FUNCTION_NAME(Type, Event, State) + catch throw:#alert{}=Alert -> + ssl_gen_statem:handle_own_alert(Alert, ?FUNCTION_NAME, State) + end. %%TODO does this make sense for DTLS ? %%-------------------------------------------------------------------- @@ -558,7 +599,12 @@ connection(Type, Event, State) -> downgrade(enter, _, State) -> {keep_state, State}; downgrade(Type, Event, State) -> - tls_dtls_connection:?FUNCTION_NAME(Type, Event, State). + try + tls_dtls_connection:?FUNCTION_NAME(Type, Event, State) + catch throw:#alert{}=Alert -> + ssl_gen_statem:handle_own_alert(Alert, ?FUNCTION_NAME, State) + end. + %%-------------------------------------------------------------------- %% gen_statem callbacks @@ -620,12 +666,15 @@ initial_state(Role, Host, Port, Socket, flight_buffer = dtls_gen_connection:new_flight(), protocol_specific = #{active_n => InternalActiveN, active_n_toggle => true, - flight_state => dtls_gen_connection:initial_flight_state(DataTag)} + flight_state => dtls_gen_connection:initial_flight_state(DataTag), + ignored_alerts => 0, + max_ignored_alerts => 10 + } }. handle_client_hello(#client_hello{client_version = ClientVersion} = Hello, State0) -> - case tls_dtls_connection:handle_sni_extension(State0, Hello) of + try #state{connection_states = ConnectionStates0, static_env = #static_env{trackers = Trackers}, handshake_env = #handshake_env{kex_algorithm = KeyExAlg, @@ -633,31 +682,28 @@ handle_client_hello(#client_hello{client_version = ClientVersion} = Hello, State negotiated_protocol = CurrentProtocol} = HsEnv, connection_env = CEnv, session = #session{own_certificates = OwnCerts} = Session0, - ssl_options = SslOpts} = State1 -> - SessionTracker = proplists:get_value(session_id_tracker, Trackers), - case dtls_handshake:hello(Hello, SslOpts, {SessionTracker, Session0, - ConnectionStates0, OwnCerts, KeyExAlg}, Renegotiation) of - #alert{} = Alert -> - ssl_gen_statem:handle_own_alert(Alert, ClientVersion, hello, State1); - {Version, {Type, Session}, - ConnectionStates, Protocol0, ServerHelloExt, HashSign} -> - Protocol = case Protocol0 of - undefined -> CurrentProtocol; - _ -> Protocol0 - end, - - State = prepare_flight(State0#state{connection_states = ConnectionStates, - connection_env = CEnv#connection_env{negotiated_version = Version}, - handshake_env = HsEnv#handshake_env{ - hashsign_algorithm = HashSign, - client_hello_version = ClientVersion, - negotiated_protocol = Protocol}, - session = Session}), - {next_state, hello, State, [{next_event, internal, {common_client_hello, Type, ServerHelloExt}}]} - end; - #alert{} = Alert -> - ssl_gen_statem:handle_own_alert(Alert, ClientVersion, hello, State0) - end. + ssl_options = SslOpts} = + tls_dtls_connection:handle_sni_extension(State0, Hello), + SessionTracker = proplists:get_value(session_id_tracker, Trackers), + {Version, {Type, Session}, ConnectionStates, Protocol0, ServerHelloExt, HashSign} = + dtls_handshake:hello(Hello, SslOpts, {SessionTracker, Session0, + ConnectionStates0, OwnCerts, KeyExAlg}, Renegotiation), + Protocol = case Protocol0 of + undefined -> CurrentProtocol; + _ -> Protocol0 + end, + + State = prepare_flight(State0#state{connection_states = ConnectionStates, + connection_env = CEnv#connection_env{negotiated_version = Version}, + handshake_env = HsEnv#handshake_env{ + hashsign_algorithm = HashSign, + client_hello_version = ClientVersion, + negotiated_protocol = Protocol}, + session = Session}), + {next_state, hello, State, [{next_event, internal, {common_client_hello, Type, ServerHelloExt}}]} + catch #alert{} = Alert -> + alert_or_reset_connection(Alert, hello, State0) + end. handle_state_timeout(flight_retransmission_timeout, StateName, @@ -670,40 +716,43 @@ handle_state_timeout(flight_retransmission_timeout, StateName, %% This will reset the retransmission timer by repeating the enter state event {repeat_state, State, Actions}. +alert_or_reset_connection(Alert, StateName, #state{connection_states = Cs} = State) -> + case maps:get(previous_cs, Cs, undefined) of + undefined -> + ssl_gen_statem:handle_own_alert(Alert, StateName, State); + PreviousConn -> + %% There exists an old connection and the new one failed, + %% reset to the old working one. + %% The next alert will be sent + HsEnv0 = State#state.handshake_env, + HsEnv = HsEnv0#handshake_env{renegotiation = undefined}, + NewState = State#state{connection_states = PreviousConn, + handshake_env = HsEnv + }, + {next_state, connection, NewState} + end. - -gen_handshake(StateName, Type, Event, - #state{connection_env = #connection_env{negotiated_version = Version}} = State) -> - try tls_dtls_connection:StateName(Type, Event, State) of - Result -> - Result - catch - _:_ -> - ssl_gen_statem:handle_own_alert(?ALERT_REC(?FATAL, ?HANDSHAKE_FAILURE, - malformed_handshake_data), - Version, StateName, State) +gen_handshake(StateName, Type, Event, State) -> + try tls_dtls_connection:StateName(Type, Event, State) + catch + throw:#alert{}=Alert -> + alert_or_reset_connection(Alert, StateName, State); + error:_ -> + Alert = ?ALERT_REC(?FATAL, ?HANDSHAKE_FAILURE, malformed_handshake_data), + alert_or_reset_connection(Alert, StateName, State) end. -gen_info(Event, connection = StateName, #state{connection_env = #connection_env{negotiated_version = Version}} = State) -> - try dtls_gen_connection:handle_info(Event, StateName, State) of - Result -> - Result - catch - _:_ -> - ssl_gen_statem:handle_own_alert(?ALERT_REC(?FATAL, ?INTERNAL_ERROR, - malformed_data), - Version, StateName, State) +gen_info(Event, connection = StateName, State) -> + try dtls_gen_connection:handle_info(Event, StateName, State) + catch error:_ -> + Alert = ?ALERT_REC(?FATAL, ?INTERNAL_ERROR, malformed_data), + alert_or_reset_connection(Alert, StateName, State) end; - -gen_info(Event, StateName, #state{connection_env = #connection_env{negotiated_version = Version}} = State) -> - try dtls_gen_connection:handle_info(Event, StateName, State) of - Result -> - Result - catch - _:_ -> - ssl_gen_statem:handle_own_alert(?ALERT_REC(?FATAL, ?HANDSHAKE_FAILURE, - malformed_handshake_data), - Version, StateName, State) +gen_info(Event, StateName, State) -> + try dtls_gen_connection:handle_info(Event, StateName, State) + catch error:_ -> + Alert = ?ALERT_REC(?FATAL, ?HANDSHAKE_FAILURE,malformed_handshake_data), + alert_or_reset_connection(Alert, StateName, State) end. prepare_flight(#state{flight_buffer = Flight, @@ -763,7 +812,7 @@ send_application_data(Data, From, _StateName, {Msgs, ConnectionStates} = dtls_record:encode_data(Data, Version, ConnectionStates0), State = State0#state{connection_states = ConnectionStates}, - case dtls_gen_connection:send(Transport, Socket, Msgs) of + case send_msgs(Transport, Socket, Msgs) of ok -> ssl_logger:debug(LogLevel, outbound, 'record', Msgs), ssl_gen_statem:hibernate_after(connection, State, [{reply, From, ok}]); @@ -772,6 +821,14 @@ send_application_data(Data, From, _StateName, end end. +send_msgs(Transport, Socket, [Msg|Msgs]) -> + case dtls_gen_connection:send(Transport, Socket, Msg) of + ok -> send_msgs(Transport, Socket, Msgs); + Error -> Error + end; +send_msgs(_, _, []) -> + ok. + time_to_renegotiate(_Data, #{current_write := #{sequence_number := Num}}, RenegotiateAt) -> diff --git a/lib/ssl/src/dtls_connection_sup.erl b/lib/ssl/src/dtls_connection_sup.erl index 4c5c0a490f..b2b9708209 100644 --- a/lib/ssl/src/dtls_connection_sup.erl +++ b/lib/ssl/src/dtls_connection_sup.erl @@ -51,17 +51,17 @@ start_child_dist(Args) -> %%%========================================================================= %%% Supervisor callback %%%========================================================================= -init(_O) -> - RestartStrategy = simple_one_for_one, - MaxR = 0, - MaxT = 3600, - - Name = undefined, % As simple_one_for_one is used. - StartFunc = {ssl_gen_statem, start_link, []}, - Restart = temporary, % E.g. should not be restarted - Shutdown = 4000, - Modules = [ssl_gen_statem, dtls_connection], - Type = worker, - - ChildSpec = {Name, StartFunc, Restart, Shutdown, Type, Modules}, - {ok, {{RestartStrategy, MaxR, MaxT}, [ChildSpec]}}. +init(_) -> + SupFlags = #{strategy => simple_one_for_one, + intensity => 0, + period => 3600 + }, + ChildSpecs = [#{id => undefined, + start => {ssl_gen_statem, start_link, []}, + restart => temporary, + shutdown => 4000, + modules => [ssl_gen_statem, dtls_connection], + type => worker + } + ], + {ok, {SupFlags, ChildSpecs}}. diff --git a/lib/ssl/src/dtls_gen_connection.erl b/lib/ssl/src/dtls_gen_connection.erl index 2032d77074..dab05335c0 100644 --- a/lib/ssl/src/dtls_gen_connection.erl +++ b/lib/ssl/src/dtls_gen_connection.erl @@ -68,6 +68,14 @@ send_alert_in_connection/2, close/5, protocol_name/0]). + + +%% See thread @ http://lists.cluenet.de/pipermail/ipv6-ops/2011-June/005755.html +%% 1280 - headers +-define(PMTUEstimate, 1200). + + + %%==================================================================== %% Internal application API %%==================================================================== @@ -105,30 +113,29 @@ next_record(#state{protocol_buffers = CurrentRead = dtls_record:get_connection_state_by_epoch(Epoch, ConnectionStates, read), case dtls_record:replay_detect(CT, CurrentRead) of false -> - decode_cipher_text(State#state{connection_states = ConnectionStates}) ; + decode_cipher_text(State) ; true -> %% Ignore replayed record - next_record(State#state{protocol_buffers = - Buffers#protocol_buffers{dtls_cipher_texts = Rest}, - connection_states = ConnectionStates}) + next_record(State#state{protocol_buffers = Buffers#protocol_buffers{dtls_cipher_texts = Rest}}) end; next_record(#state{protocol_buffers = #protocol_buffers{dtls_cipher_texts = [#ssl_tls{epoch = Epoch} | Rest]} = Buffers, - connection_states = #{current_read := #{epoch := CurrentEpoch}} = ConnectionStates} = State) + connection_states = #{current_read := #{epoch := CurrentEpoch}}} = State) when Epoch > CurrentEpoch -> %% TODO Buffer later Epoch message, drop it for now - next_record(State#state{protocol_buffers = - Buffers#protocol_buffers{dtls_cipher_texts = Rest}, - connection_states = ConnectionStates}); -next_record(#state{protocol_buffers = - #protocol_buffers{dtls_cipher_texts = [ _ | Rest]} - = Buffers, - connection_states = ConnectionStates} = State) -> - %% Drop old epoch message - next_record(State#state{protocol_buffers = - Buffers#protocol_buffers{dtls_cipher_texts = Rest}, - connection_states = ConnectionStates}); + next_record(State#state{protocol_buffers = Buffers#protocol_buffers{dtls_cipher_texts = Rest}}); +next_record(#state{protocol_buffers = #protocol_buffers{dtls_cipher_texts = + [#ssl_tls{epoch = Epoch} | Rest] + } = Buffers + } = State) -> + case Epoch of + 0 -> %% A reconnect (client might have rebooted and re-connected) + decode_cipher_text(State); + _ -> + %% Drop old epoch message + next_record(State#state{protocol_buffers = Buffers#protocol_buffers{dtls_cipher_texts = Rest}}) + end; next_record(#state{static_env = #static_env{role = server, socket = {Listener, {Client, _}}}} = State) -> dtls_packet_demux:active_once(Listener, Client, self()), @@ -182,16 +189,16 @@ next_event(StateName, no_record, %% TODO maybe buffer later epoch next_event(StateName, no_record, State, Actions); {#alert{} = Alert, State} -> - Version = State#state.connection_env#connection_env.negotiated_version, - handle_own_alert(Alert, Version, StateName, State) + handle_own_alert(Alert, StateName, State) end; next_event(connection = StateName, Record, #state{connection_states = #{current_read := #{epoch := CurrentEpoch}}} = State0, Actions) -> case Record of - #ssl_tls{epoch = CurrentEpoch, + #ssl_tls{epoch = Epoch, type = ?HANDSHAKE, - version = Version} = Record -> - State = dtls_version(StateName, Version, State0), + version = Version} = Record + when Epoch =:= CurrentEpoch; Epoch =:= 0 -> + State = dtls_version(StateName, Version, State0), {next_state, StateName, State, [{next_event, internal, {protocol_record, Record}} | Actions]}; #ssl_tls{epoch = CurrentEpoch} -> @@ -223,8 +230,7 @@ next_event(StateName, Record, %% TODO maybe buffer later epoch next_event(StateName, no_record, State0, Actions); #alert{} = Alert -> - Version = State0#state.connection_env#connection_env.negotiated_version, - handle_own_alert(Alert, Version, StateName, State0) + handle_own_alert(Alert, StateName, State0) end. initial_flight_state(udp)-> @@ -246,12 +252,11 @@ send_handshake_flight(#state{static_env = #static_env{socket = Socket, connection_states = ConnectionStates0, ssl_options = #{log_level := LogLevel}} = State0, Epoch) -> - PMTUEstimate = 1400, %% TODO make configurable #{current_write := #{max_fragment_length := MaxFragmentLength}} = ConnectionStates0, - MaxSize = min(MaxFragmentLength, PMTUEstimate), + MaxSize = min(MaxFragmentLength, ?PMTUEstimate), {Encoded, ConnectionStates} = encode_handshake_flight(lists:reverse(Flight), Version, MaxSize, Epoch, ConnectionStates0), - send(Transport, Socket, Encoded), + send_packets(Transport, Socket, Encoded), ssl_logger:debug(LogLevel, outbound, 'record', Encoded), {State0#state{connection_states = ConnectionStates}, []}; @@ -264,14 +269,12 @@ send_handshake_flight(#state{static_env = #static_env{socket = Socket, connection_states = ConnectionStates0, ssl_options = #{log_level := LogLevel}} = State0, Epoch) -> - PMTUEstimate = 1400, %% TODO make configurable #{current_write := #{max_fragment_length := MaxFragmentLength}} = ConnectionStates0, - MaxSize = min(MaxFragmentLength, PMTUEstimate), + MaxSize = min(MaxFragmentLength, ?PMTUEstimate), {HsBefore, ConnectionStates1} = encode_handshake_flight(lists:reverse(Flight0), Version, MaxSize, Epoch, ConnectionStates0), {EncChangeCipher, ConnectionStates} = encode_change_cipher(ChangeCipher, Version, Epoch, ConnectionStates1), - - send(Transport, Socket, [HsBefore, EncChangeCipher]), + send_packets(Transport, Socket, HsBefore ++ EncChangeCipher), ssl_logger:debug(LogLevel, outbound, 'record', [HsBefore]), ssl_logger:debug(LogLevel, outbound, 'record', [EncChangeCipher]), {State0#state{connection_states = ConnectionStates}, []}; @@ -285,16 +288,15 @@ send_handshake_flight(#state{static_env = #static_env{socket = Socket, connection_states = ConnectionStates0, ssl_options = #{log_level := LogLevel}} = State0, Epoch) -> - PMTUEstimate = 1400, %% TODO make configurable #{current_write := #{max_fragment_length := MaxFragmentLength}} = ConnectionStates0, - MaxSize = min(MaxFragmentLength, PMTUEstimate), + MaxSize = min(MaxFragmentLength, ?PMTUEstimate), {HsBefore, ConnectionStates1} = encode_handshake_flight(lists:reverse(Flight0), Version, MaxSize, Epoch-1, ConnectionStates0), {EncChangeCipher, ConnectionStates2} = encode_change_cipher(ChangeCipher, Version, Epoch-1, ConnectionStates1), {HsAfter, ConnectionStates} = encode_handshake_flight(lists:reverse(Flight1), Version, MaxSize, Epoch, ConnectionStates2), - send(Transport, Socket, [HsBefore, EncChangeCipher, HsAfter]), + send_packets(Transport, Socket, HsBefore ++ EncChangeCipher ++ HsAfter), ssl_logger:debug(LogLevel, outbound, 'record', [HsBefore]), ssl_logger:debug(LogLevel, outbound, 'record', [EncChangeCipher]), ssl_logger:debug(LogLevel, outbound, 'record', [HsAfter]), @@ -309,14 +311,13 @@ send_handshake_flight(#state{static_env = #static_env{socket = Socket, connection_states = ConnectionStates0, ssl_options = #{log_level := LogLevel}} = State0, Epoch) -> - PMTUEstimate = 1400, %% TODO make configurable #{current_write := #{max_fragment_length := MaxFragmentLength}} = ConnectionStates0, - MaxSize = min(MaxFragmentLength, PMTUEstimate), + MaxSize = min(MaxFragmentLength, ?PMTUEstimate), {EncChangeCipher, ConnectionStates1} = encode_change_cipher(ChangeCipher, Version, Epoch-1, ConnectionStates0), {HsAfter, ConnectionStates} = encode_handshake_flight(lists:reverse(Flight1), Version, MaxSize, Epoch, ConnectionStates1), - send(Transport, Socket, [EncChangeCipher, HsAfter]), + send_packets(Transport, Socket, EncChangeCipher ++ HsAfter), ssl_logger:debug(LogLevel, outbound, 'record', [EncChangeCipher]), ssl_logger:debug(LogLevel, outbound, 'record', [HsAfter]), {State0#state{connection_states = ConnectionStates}, []}. @@ -332,9 +333,8 @@ handle_protocol_record(#ssl_tls{type = ?APPLICATION_DATA, fragment = Data}, Stat ssl_gen_statem:hibernate_after(StateName, State, Actions) end; %%% DTLS record protocol level handshake messages -handle_protocol_record(#ssl_tls{type = ?HANDSHAKE, - fragment = Data}, - StateName, +handle_protocol_record(#ssl_tls{type = ?HANDSHAKE, epoch = Epoch, fragment = Data}, + StateName, #state{protocol_buffers = Buffers0, connection_env = #connection_env{negotiated_version = Version}, ssl_options = Options} = State) -> @@ -344,27 +344,31 @@ handle_protocol_record(#ssl_tls{type = ?HANDSHAKE, next_event(StateName, no_record, State#state{protocol_buffers = Buffers}); {Packets, Buffers} -> HsEnv = State#state.handshake_env, - Events = dtls_handshake_events(Packets), - {next_state, StateName, + HSEvents = dtls_handshake_events(Packets), + Events = case is_new_connection(Epoch, Packets, State) of + true -> [{next_event, internal, new_connection} | HSEvents]; + false -> HSEvents + end, + {next_state, StateName, State#state{protocol_buffers = Buffers, - handshake_env = - HsEnv#handshake_env{unprocessed_handshake_events - = unprocessed_events(Events)}}, Events} + handshake_env = + HsEnv#handshake_env{ + unprocessed_handshake_events = unprocessed_events(HSEvents)} + }, Events} end catch throw:#alert{} = Alert -> - handle_own_alert(Alert, Version, StateName, State) + handle_own_alert(Alert, StateName, State) end; %%% DTLS record protocol level change cipher messages handle_protocol_record(#ssl_tls{type = ?CHANGE_CIPHER_SPEC, fragment = Data}, StateName, State) -> {next_state, StateName, State, [{next_event, internal, #change_cipher_spec{type = Data}}]}; %%% DTLS record protocol level Alert messages -handle_protocol_record(#ssl_tls{type = ?ALERT, fragment = EncAlerts}, StateName, - #state{connection_env = #connection_env{negotiated_version = Version}} = State) -> +handle_protocol_record(#ssl_tls{type = ?ALERT, fragment = EncAlerts}, StateName, State) -> case decode_alerts(EncAlerts) of Alerts = [_|_] -> handle_alerts(Alerts, {next_state, StateName, State}); #alert{} = Alert -> - handle_own_alert(Alert, Version, StateName, State) + handle_own_alert(Alert, StateName, State) end; %% Ignore unknown TLS record level protocol messages handle_protocol_record(#ssl_tls{type = _Unknown}, StateName, State) -> @@ -478,6 +482,28 @@ send(Transport, {Listener, Socket}, Data) when is_pid(Listener) -> send(Transport, Socket, Data) -> % Client socket dtls_socket:send(Transport, Socket, Data). +send_packets(_Transport, _Socket, []) -> + ok; +send_packets(Transport, Socket, Packets) -> + {Packet, Rest} = pack_packets(Packets, 0, ?PMTUEstimate+80, []), + case send(Transport, Socket, Packet) of + ok -> send_packets(Transport, Socket, Rest); + Err -> Err + end. + +pack_packets([P|Rest]=Packets, SoFar, Max, Acc) -> + Size = erlang:iolist_size(P), + Next = SoFar + Size, + if Size > Max, Acc =:= [] -> + {P, Rest}; + Next < Max -> + pack_packets(Rest, Next, Max, [P|Acc]); + true -> + {lists:reverse(Acc), Packets} + end; +pack_packets([], _, _, Acc) -> + {lists:reverse(Acc), []}. + socket(Pid, Transport, Socket, _Tracker) -> dtls_socket:socket(Pid, Transport, Socket, ?MODULE). @@ -489,14 +515,12 @@ getopts(Transport, Socket, Tag) -> %% raw data from socket, unpack records handle_info({Protocol, _, _, _, Data}, StateName, - #state{static_env = #static_env{role = Role, - data_tag = Protocol}} = State0) -> + #state{static_env = #static_env{data_tag = Protocol}} = State0) -> case next_dtls_record(Data, StateName, State0) of {Record, State} -> next_event(StateName, Record, State); #alert{} = Alert -> - ssl_gen_statem:handle_normal_shutdown(Alert#alert{role = Role}, StateName, State0), - {stop, {shutdown, own_alert}, State0} + handle_own_alert(Alert, StateName, State0) end; handle_info({PassiveTag, Socket}, StateName, @@ -555,6 +579,15 @@ handle_info(Msg, StateName, State) -> %% Internal functions %%==================================================================== +is_new_connection(0, [{#client_hello{},_Raw}|_], + #state{ + connection_states = + #{current_read := #{epoch := CurrentEpoch}}}) + when CurrentEpoch > 0 -> + true; +is_new_connection(_, _, _) -> + false. + dtls_handshake_events(Packets) -> lists:map(fun(Packet) -> {next_event, internal, {handshake, Packet}} @@ -569,10 +602,15 @@ unprocessed_events(Events) -> erlang:length(Events)-1. encode_handshake_flight(Flight, Version, MaxFragmentSize, Epoch, ConnectionStates) -> - Fragments = lists:map(fun(Handshake) -> - dtls_handshake:fragment_handshake(Handshake, MaxFragmentSize) - end, Flight), - dtls_record:encode_handshake(Fragments, Version, Epoch, ConnectionStates). + Encode = fun(Fragment, {Acc, Cs0}) -> + {Enc, Cs} = dtls_record:encode_handshake(Fragment, Version, Epoch, Cs0), + {[Enc|Acc], Cs} + end, + {Rev, Cs} = lists:foldl(fun(Handshake, {Acc,Cs0}) -> + Frags = dtls_handshake:fragment_handshake(Handshake, MaxFragmentSize), + lists:foldl(Encode, {Acc,Cs0}, Frags) + end, {[], ConnectionStates}, Flight), + {lists:reverse(Rev), Cs}. encode_change_cipher(#change_cipher_spec{}, Version, Epoch, ConnectionStates) -> dtls_record:encode_change_cipher_spec(Version, Epoch, ConnectionStates). @@ -621,12 +659,28 @@ handle_alerts([], Result) -> Result; handle_alerts(_, {stop, _, _} = Stop) -> Stop; -handle_alerts([Alert | Alerts], {next_state, StateName, State}) -> - handle_alerts(Alerts, ssl_gen_statem:handle_alert(Alert, StateName, State)); -handle_alerts([Alert | Alerts], {next_state, StateName, State, _Actions}) -> - handle_alerts(Alerts, ssl_gen_statem:handle_alert(Alert, StateName, State)). +handle_alerts(Alerts, {next_state, StateName, State}) -> + handle_alerts_or_reset(Alerts, StateName, State); +handle_alerts(Alerts, {next_state, StateName, State, _Actions}) -> + handle_alerts_or_reset(Alerts, StateName, State). + +handle_alerts_or_reset([Alert|Alerts], StateName, #state{connection_states = Cs} = State) -> + case maps:get(previous_cs, Cs, undefined) of + undefined -> + handle_alerts(Alerts, ssl_gen_statem:handle_alert(Alert, StateName, State)); + PreviousConn -> + %% There exists an old connection and the new one sent alerts, + %% reset to the old working one. + HsEnv0 = State#state.handshake_env, + HsEnv = HsEnv0#handshake_env{renegotiation = undefined}, + NewState = State#state{connection_states = PreviousConn, + handshake_env = HsEnv + }, + {next_state, connection, NewState} + end. + -handle_own_alert(Alert, Version, StateName, +handle_own_alert(Alert, StateName, #state{static_env = #static_env{data_tag = udp, role = Role}, ssl_options = #{log_level := LogLevel}} = State0) -> @@ -635,10 +689,11 @@ handle_own_alert(Alert, Version, StateName, log_ignore_alert(LogLevel, StateName, Alert, Role), {next_state, StateName, State}; {false, State} -> - ssl_gen_statem:handle_own_alert(Alert, Version, StateName, State) + dtls_connection:alert_or_reset_connection(Alert, StateName, State) end; -handle_own_alert(Alert, Version, StateName, State) -> - ssl_gen_statem:handle_own_alert(Alert, Version, StateName, State). +handle_own_alert(Alert, StateName, State) -> + dtls_connection:alert_or_reset_connection(Alert, StateName, State). + ignore_alert(#alert{level = ?FATAL}, #state{protocol_specific = #{ignored_alerts := N, max_ignored_alerts := N}} = State) -> {false, State}; diff --git a/lib/ssl/src/dtls_handshake.erl b/lib/ssl/src/dtls_handshake.erl index 9999933d90..e46a52fb8c 100644 --- a/lib/ssl/src/dtls_handshake.erl +++ b/lib/ssl/src/dtls_handshake.erl @@ -108,7 +108,7 @@ hello(#server_hello{server_version = Version, random = Random, Compression, HelloExt, SslOpt, ConnectionStates0, Renegotiation, IsNew); false -> - ?ALERT_REC(?FATAL, ?PROTOCOL_VERSION) + throw(?ALERT_REC(?FATAL, ?PROTOCOL_VERSION)) end. hello(#client_hello{client_version = ClientVersion} = Hello, #{versions := Versions} = SslOpts, @@ -155,7 +155,7 @@ encode_handshake(Handshake, Version, Seq) -> %%-------------------------------------------------------------------- -spec get_dtls_handshake(ssl_record:ssl_version(), binary(), #protocol_buffers{}, ssl_options()) -> - {[dtls_handshake()], #protocol_buffers{}}. + {[{dtls_handshake(), binary()}], #protocol_buffers{}}. %% %% Description: Given buffered and new data from dtls_record, collects %% and returns it as a list of handshake messages, also returns @@ -195,13 +195,13 @@ handle_client_hello(Version, SslOpts, OwnCert), case CipherSuite of no_suite -> - ?ALERT_REC(?FATAL, ?INSUFFICIENT_SECURITY); + throw(?ALERT_REC(?FATAL, ?INSUFFICIENT_SECURITY)); _ -> #{key_exchange := KeyExAlg} = ssl_cipher_format:suite_bin_to_map(CipherSuite), case ssl_handshake:select_hashsign({ClientHashSigns, undefined}, OwnCert, KeyExAlg, SupportedHashSigns, TLSVersion) of #alert{} = Alert -> - Alert; + throw(Alert); HashSign -> handle_client_hello_extensions(Version, Type, Random, CipherSuites, HelloExt, SslOpts, Session1, ConnectionStates0, @@ -209,33 +209,27 @@ handle_client_hello(Version, end end; false -> - ?ALERT_REC(?FATAL, ?PROTOCOL_VERSION) + throw(?ALERT_REC(?FATAL, ?PROTOCOL_VERSION)) end. handle_client_hello_extensions(Version, Type, Random, CipherSuites, HelloExt, SslOpts, Session0, ConnectionStates0, Renegotiation, HashSign) -> - try ssl_handshake:handle_client_hello_extensions(dtls_record, Random, CipherSuites, - HelloExt, dtls_v1:corresponding_tls_version(Version), - SslOpts, Session0, + {Session, ConnectionStates, Protocol, ServerHelloExt} = + ssl_handshake:handle_client_hello_extensions(dtls_record, Random, CipherSuites, + HelloExt, dtls_v1:corresponding_tls_version(Version), + SslOpts, Session0, ConnectionStates0, Renegotiation, - Session0#session.is_resumable) of - {Session, ConnectionStates, Protocol, ServerHelloExt} -> - {Version, {Type, Session}, ConnectionStates, Protocol, ServerHelloExt, HashSign} - catch throw:Alert -> - Alert - end. + Session0#session.is_resumable), + {Version, {Type, Session}, ConnectionStates, Protocol, ServerHelloExt, HashSign}. handle_server_hello_extensions(Version, SessionId, Random, CipherSuite, Compression, HelloExt, SslOpt, ConnectionStates0, Renegotiation, IsNew) -> - try ssl_handshake:handle_server_hello_extensions(dtls_record, Random, CipherSuite, + {ConnectionStates, ProtoExt, Protocol, OcspState} = + ssl_handshake:handle_server_hello_extensions(dtls_record, Random, CipherSuite, Compression, HelloExt, dtls_v1:corresponding_tls_version(Version), - SslOpt, ConnectionStates0, Renegotiation, IsNew) of - {ConnectionStates, ProtoExt, Protocol, OcspState} -> - {Version, SessionId, ConnectionStates, ProtoExt, Protocol, OcspState} - catch throw:Alert -> - Alert - end. + SslOpt, ConnectionStates0, Renegotiation, IsNew), + {Version, SessionId, ConnectionStates, ProtoExt, Protocol, OcspState}. %%-------------------------------------------------------------------- diff --git a/lib/ssl/src/dtls_listener_sup.erl b/lib/ssl/src/dtls_listener_sup.erl index 4f46407290..699f27b95f 100644 --- a/lib/ssl/src/dtls_listener_sup.erl +++ b/lib/ssl/src/dtls_listener_sup.erl @@ -76,18 +76,17 @@ register_listener(OwnerAndListner, IP, Port) -> %%%========================================================================= %%% Supervisor callback %%%========================================================================= -init(_O) -> - ets:new(dtls_listener_sup, [named_table, public, set]), - RestartStrategy = simple_one_for_one, - MaxR = 0, - MaxT = 3600, - - Name = undefined, % As simple_one_for_one is used. - StartFunc = {dtls_packet_demux, start_link, []}, - Restart = temporary, % E.g. should not be restarted - Shutdown = 4000, - Modules = [dtls_packet_demux], - Type = worker, - - ChildSpec = {Name, StartFunc, Restart, Shutdown, Type, Modules}, - {ok, {{RestartStrategy, MaxR, MaxT}, [ChildSpec]}}. +init(_) -> + ets:new(dtls_listener_sup, [named_table, public, set]), + SupFlags = #{strategy => simple_one_for_one, + intensity => 0, + period => 3600 + }, + ChildSpecs = [#{id => undefined, + start => {dtls_packet_demux, start_link, []}, + restart => temporary, + shutdown => 4000, + modules => [dtls_packet_demux], + type => worker + }], + {ok, {SupFlags, ChildSpecs}}. diff --git a/lib/ssl/src/dtls_record.erl b/lib/ssl/src/dtls_record.erl index ef275fad4c..e3c9b829c4 100644 --- a/lib/ssl/src/dtls_record.erl +++ b/lib/ssl/src/dtls_record.erl @@ -75,6 +75,9 @@ init_connection_states(Role, BeastMitigation) -> ConnectionEnd = ssl_record:record_protocol_role(Role), Initial = initial_connection_state(ConnectionEnd, BeastMitigation), Current = Initial#{epoch := 0}, + %% No need to pass Version to ssl_record:empty_connection_state since + %% random nonce is generated with same algorithm for DTLS version + %% Might require a change for DTLS-1.3 InitialPending = ssl_record:empty_connection_state(ConnectionEnd, BeastMitigation), Pending = empty_connection_state(InitialPending), #{saved_read => Current, @@ -204,16 +207,17 @@ encode_alert_record(#alert{level = Level, description = Description}, %%-------------------------------------------------------------------- -spec encode_change_cipher_spec(ssl_record:ssl_version(), integer(), ssl_record:connection_states()) -> - {iolist(), ssl_record:connection_states()}. + {[iolist()], ssl_record:connection_states()}. %% %% Description: Encodes a change_cipher_spec-message to send on the ssl socket. %%-------------------------------------------------------------------- encode_change_cipher_spec(Version, Epoch, ConnectionStates) -> - encode_plain_text(?CHANGE_CIPHER_SPEC, Version, Epoch, ?byte(?CHANGE_CIPHER_SPEC_PROTO), ConnectionStates). + {Enc, Cs} = encode_plain_text(?CHANGE_CIPHER_SPEC, Version, Epoch, ?byte(?CHANGE_CIPHER_SPEC_PROTO), ConnectionStates), + {[Enc], Cs}. %%-------------------------------------------------------------------- -spec encode_data(binary(), ssl_record:ssl_version(), ssl_record:connection_states()) -> - {iolist(),ssl_record:connection_states()}. + {[iolist()],ssl_record:connection_states()}. %% %% Description: Encodes data to send on the ssl-socket. %%-------------------------------------------------------------------- @@ -236,7 +240,8 @@ encode_data(Data, Version, ConnectionStates) -> end, {[], ConnectionStates}, Frags), {lists:reverse(RevCipherText), ConnectionStates1}; _ -> - encode_plain_text(?APPLICATION_DATA, Version, Epoch, Data, ConnectionStates) + {Enc, Cs} = encode_plain_text(?APPLICATION_DATA, Version, Epoch, Data, ConnectionStates), + {[Enc], Cs} end. encode_plain_text(Type, Version, Epoch, Data, ConnectionStates) -> @@ -455,7 +460,8 @@ get_dtls_records_aux({_, _, Version, _} = Vinfo, <<?BYTE(Type),?BYTE(MajVer),?BY ?ALERT_REC(?FATAL, ?BAD_RECORD_MAC) end; get_dtls_records_aux(_, <<?BYTE(_), ?BYTE(_MajVer), ?BYTE(_MinVer), - ?UINT16(Length), _/binary>>, + ?UINT16(_Epoch), ?UINT48(_Seq), + ?UINT16(Length), _/binary>>, _Acc, _) when Length > ?MAX_CIPHER_TEXT_LENGTH -> ?ALERT_REC(?FATAL, ?RECORD_OVERFLOW); diff --git a/lib/ssl/src/dtls_server_session_cache_sup.erl b/lib/ssl/src/dtls_server_session_cache_sup.erl index 65fbb34918..c0242ab2c0 100644 --- a/lib/ssl/src/dtls_server_session_cache_sup.erl +++ b/lib/ssl/src/dtls_server_session_cache_sup.erl @@ -47,17 +47,16 @@ start_child(Listener) -> %%%========================================================================= %%% Supervisor callback %%%========================================================================= -init(_O) -> - RestartStrategy = simple_one_for_one, - MaxR = 0, - MaxT = 3600, - - Name = undefined, % As simple_one_for_one is used. - StartFunc = {ssl_server_session_cache, start_link, []}, - Restart = temporary, % E.g. should not be restarted - Shutdown = 4000, - Modules = [ssl_server_session_cache], - Type = worker, - - ChildSpec = {Name, StartFunc, Restart, Shutdown, Type, Modules}, - {ok, {{RestartStrategy, MaxR, MaxT}, [ChildSpec]}}. +init(_) -> + SupFlags = #{strategy => simple_one_for_one, + intensity => 0, + period => 3600 + }, + ChildSpecs = [#{id => undefined, + start => {ssl_server_session_cache, start_link, []}, + restart => temporary, + shutdown => 4000, + modules => [ssl_server_session_cache], + type => worker + }], + {ok, {SupFlags, ChildSpecs}}. diff --git a/lib/ssl/src/dtls_server_sup.erl b/lib/ssl/src/dtls_server_sup.erl index 7ec6db3984..1430627cf2 100644 --- a/lib/ssl/src/dtls_server_sup.erl +++ b/lib/ssl/src/dtls_server_sup.erl @@ -43,33 +43,34 @@ start_link() -> %%% Supervisor callback %%%========================================================================= -init([]) -> - DTLSListeners = dtls_listeners_spec(), - %% Add SessionTracker if we add DTLS-1.3 - Pre_1_3SessionTracker = ssl_server_session_child_spec(), - - {ok, {{one_for_all, 10, 3600}, [DTLSListeners, - Pre_1_3SessionTracker - ]}}. - +init([]) -> + SupFlags = #{strategy => one_for_all, + intensity => 10, + period => 3600 + }, + ChildSpecs = [dtls_listeners_spec(), + ssl_server_session_child_spec() + %% TODO Add DTLS-1.3 session ticket handling + ], + {ok, {SupFlags, ChildSpecs}}. %%-------------------------------------------------------------------- %%% Internal functions %%-------------------------------------------------------------------- dtls_listeners_spec() -> - Name = dtls_listener, - StartFunc = {dtls_listener_sup, start_link, []}, - Restart = permanent, - Shutdown = 4000, - Modules = [], - Type = supervisor, - {Name, StartFunc, Restart, Shutdown, Type, Modules}. + #{id => dtls_listener_sup, + start => {dtls_listener_sup, start_link, []}, + restart => permanent, + shutdown => 4000, + modules => [dtls_listener_sup], + type => supervisor + }. ssl_server_session_child_spec() -> - Name = dtls_server_session_cache_sup, - StartFunc = {dtls_server_session_cache_sup, start_link, []}, - Restart = permanent, - Shutdown = 4000, - Modules = [dtls_server_session_cache_sup], - Type = supervisor, - {Name, StartFunc, Restart, Shutdown, Type, Modules}. + #{id => dtls_server_session_cache_sup, + start => {dtls_server_session_cache_sup, start_link, []}, + restart => permanent, + shutdown => 4000, + modules => [dtls_server_session_cache_sup], + type => supervisor + }. diff --git a/lib/ssl/src/dtls_sup.erl b/lib/ssl/src/dtls_sup.erl index acc4415a9f..2b73f13e19 100644 --- a/lib/ssl/src/dtls_sup.erl +++ b/lib/ssl/src/dtls_sup.erl @@ -44,33 +44,30 @@ start_link() -> %%%========================================================================= init([]) -> - DTLSConnectionManager = dtls_connection_manager_child_spec(), - DTLSServers = dtls_server_spec(), - - {ok, {{one_for_one, 10, 3600}, [DTLSConnectionManager, - DTLSServers - ]}}. + SupFlags = #{strategy => one_for_one, + intensity => 10, + period => 3600 + }, + Children = [dtls_connection_child_spec(), server_instance_child_spec()], + {ok, {SupFlags, Children}}. - %%-------------------------------------------------------------------- %%% Internal functions %%-------------------------------------------------------------------- -dtls_server_spec() -> - Name = dtls_servers, - StartFunc = {dtls_server_sup, start_link, []}, - Restart = permanent, - Shutdown = 4000, - Modules = [dtls_server_sup], - Type = supervisor, - {Name, StartFunc, Restart, Shutdown, Type, Modules}. - -dtls_connection_manager_child_spec() -> - Name = dtls_connection, - StartFunc = {dtls_connection_sup, start_link, []}, - Restart = permanent, - - Shutdown = 4000, - Modules = [dtls_connection_sup], - Type = supervisor, - {Name, StartFunc, Restart, Shutdown, Type, Modules}. +dtls_connection_child_spec() -> + #{id => dtls_connection_sup, + start => {dtls_connection_sup, start_link, []}, + restart => permanent, + shutdown => 4000, + modules => [dtls_connection_sup], + type => supervisor + }. +server_instance_child_spec() -> + #{id => dtls_server_sup, + start => {dtls_server_sup, start_link, []}, + restart => permanent, + shutdown => 4000, + modules => [dtls_server_sup], + type => supervisor + }. diff --git a/lib/ssl/src/inet_tls_dist.erl b/lib/ssl/src/inet_tls_dist.erl index 5cfbeff387..cddc4f90bf 100644 --- a/lib/ssl/src/inet_tls_dist.erl +++ b/lib/ssl/src/inet_tls_dist.erl @@ -53,15 +53,7 @@ select(Node) -> gen_select(inet_tcp, Node). gen_select(Driver, Node) -> - case dist_util:split_node(Node) of - {node,_,Host} -> - case Driver:getaddr(Host) of - {ok, _} -> true; - _ -> false - end; - _ -> - false - end. + inet_tcp_dist:gen_select(Driver, Node). %% ------------------------------------------------------------ %% Get the address family that this distribution uses @@ -582,8 +574,8 @@ do_setup_connect(Driver, Kernel, Node, Address, Ip, TcpPort, Version, Type, MyNo Opts = trace(connect_options(get_ssl_options(client))), dist_util:reset_timer(Timer), case ssl:connect( - Address, TcpPort, - [binary, {active, false}, {packet, 4}, + Ip, TcpPort, + [binary, {active, false}, {packet, 4}, {server_name_indication, Address}, Driver:family(), {nodelay, true}] ++ Opts, net_kernel:connecttime()) of {ok, #sslsocket{pid = [_, DistCtrl| _]} = SslSocket} -> diff --git a/lib/ssl/src/ssl.app.src b/lib/ssl/src/ssl.app.src index d761520856..a43a84f26d 100644 --- a/lib/ssl/src/ssl.app.src +++ b/lib/ssl/src/ssl.app.src @@ -18,6 +18,7 @@ tls_server_session_ticket_sup, tls_server_session_ticket, tls_sup, + tls_dyn_connection_sup, ssl_dh_groups, %% DTLS dtls_connection, diff --git a/lib/ssl/src/ssl.erl b/lib/ssl/src/ssl.erl index 672cd6ef14..b1ea481fc7 100644 --- a/lib/ssl/src/ssl.erl +++ b/lib/ssl/src/ssl.erl @@ -1526,14 +1526,15 @@ handle_options(Transport, Socket, Opts0, Role, Host) -> %% Ensure all options are evaluated at startup SslOpts1 = add_missing_options(SslOpts0, ?RULES), - SslOpts = #{protocol := Protocol} + SslOpts2 = #{protocol := Protocol} = process_options(SslOpts1, #{}, #{role => Role, host => Host, rules => ?RULES}), - maybe_client_warn_no_verify(SslOpts, Role), + maybe_client_warn_no_verify(SslOpts2, Role), + SslOpts = maps:without([warn_verify_none], SslOpts2), %% Handle special options {Sock, Emulated} = emulated_options(Transport, Socket, Protocol, SockOpts0), ConnetionCb = connection_cb(Protocol), @@ -1862,7 +1863,7 @@ handle_option(user_lookup_fun = Option, Value0, Value = validate_option(Option, Value0), OptionsMap#{Option => Value}; handle_option(verify = Option, unbound, OptionsMap, #{rules := Rules}) -> - handle_verify_option(default_value(Option, Rules), OptionsMap); + handle_verify_option(default_value(Option, Rules), OptionsMap#{warn_verify_none => true}); handle_option(verify = _Option, Value, OptionsMap, _Env) -> handle_verify_option(Value, OptionsMap); handle_option(verify_fun = Option, unbound, #{verify := Verify} = OptionsMap, #{rules := Rules}) @@ -2830,9 +2831,11 @@ add_filter(undefined, Filters) -> add_filter(Filter, Filters) -> [Filter | Filters]. -maybe_client_warn_no_verify(#{verify := verify_none, log_level := LogLevel}, client) -> - ssl_logger:log(warning, LogLevel, #{description => "Authenticity is not established by certificate path validation", - reason => "Option {verify, verify_peer} and cacertfile/cacerts is missing"}, #{}); +maybe_client_warn_no_verify(#{verify := verify_none, + warn_verify_none := true, + log_level := LogLevel}, client) -> + ssl_logger:log(warning, LogLevel, #{description => "Authenticity is not established by certificate path validation", + reason => "Option {verify, verify_peer} and cacertfile/cacerts is missing"}, #{}); maybe_client_warn_no_verify(_,_) -> - %% Client certificate validation is optional in TLS + %% Warning not needed. Note client certificate validation is optional in TLS ok. diff --git a/lib/ssl/src/ssl_admin_sup.erl b/lib/ssl/src/ssl_admin_sup.erl index 0cf2ab6332..9c10bf7b41 100644 --- a/lib/ssl/src/ssl_admin_sup.erl +++ b/lib/ssl/src/ssl_admin_sup.erl @@ -43,11 +43,15 @@ start_link() -> %%% Supervisor callback %%%========================================================================= -init([]) -> - PEMCache = pem_cache_child_spec(), - SessionCertManager = session_and_cert_manager_child_spec(), - TicketStore = ticket_store_spec(), - {ok, {{rest_for_one, 10, 3600}, [PEMCache, SessionCertManager, TicketStore]}}. +init([]) -> + SupFlags = #{strategy => rest_for_one, + intensity => 10, + period => 3600 + }, + ChildSpecs = [pem_cache_child_spec(), + session_and_cert_manager_child_spec(), + ticket_store_spec()], + {ok, {SupFlags, ChildSpecs}}. manager_opts() -> CbOpts = case application:get_env(ssl, session_cb) of @@ -69,34 +73,33 @@ manager_opts() -> %%-------------------------------------------------------------------- pem_cache_child_spec() -> - Name = ssl_pem_cache, - StartFunc = {ssl_pem_cache, start_link, [[]]}, - Restart = permanent, - Shutdown = 4000, - Modules = [ssl_pem_cache], - Type = worker, - {Name, StartFunc, Restart, Shutdown, Type, Modules}. - + #{id => ssl_pem_cache, + start => {ssl_pem_cache, start_link, [[]]}, + restart => permanent, + shutdown => 4000, + modules => [ssl_pem_cache], + type => worker + }. session_and_cert_manager_child_spec() -> Opts = manager_opts(), - Name = ssl_manager, - StartFunc = {ssl_manager, start_link, [Opts]}, - Restart = permanent, - Shutdown = 4000, - Modules = [ssl_manager], - Type = worker, - {Name, StartFunc, Restart, Shutdown, Type, Modules}. + #{id => ssl_manager, + start => {ssl_manager, start_link, [Opts]}, + restart => permanent, + shutdown => 4000, + modules => [ssl_manager], + type => worker + }. ticket_store_spec() -> - Name = tls_client_ticket_store, Size = client_session_ticket_store_size(), Lifetime = client_session_ticket_lifetime(), - StartFunc = {tls_client_ticket_store, start_link, [Size,Lifetime]}, - Restart = permanent, - Shutdown = 4000, - Modules = [tls_client_ticket_store], - Type = worker, - {Name, StartFunc, Restart, Shutdown, Type, Modules}. + #{id => tls_client_ticket_store, + start => {tls_client_ticket_store, start_link, [Size, Lifetime]}, + restart => permanent, + shutdown => 4000, + modules => [tls_client_ticket_store], + type => worker + }. session_cb_init_args() -> case application:get_env(ssl, session_cb_init_args) of diff --git a/lib/ssl/src/ssl_alert.hrl b/lib/ssl/src/ssl_alert.hrl index eb3e3ec837..420b014aae 100644 --- a/lib/ssl/src/ssl_alert.hrl +++ b/lib/ssl/src/ssl_alert.hrl @@ -26,6 +26,7 @@ -ifndef(ssl_alert). -define(ssl_alert, true). +%%-define(ssl_debug, true). -include_lib("kernel/include/logger.hrl"). %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% @@ -114,8 +115,14 @@ -define(CERTIFICATE_REQUIRED, 116). -define(NO_APPLICATION_PROTOCOL, 120). --define(ALERT_REC(Level,Desc), #alert{level=Level,description=Desc,where= ?LOCATION}). --define(ALERT_REC(Level,Desc,Reason), #alert{level=Level,description=Desc,where=?LOCATION,reason=Reason}). +-ifdef(ssl_debug). +-define(ST_LOCATION, fun(Map) -> Map#{st => process_info(self(), current_stacktrace)} end (?LOCATION)). +-else. +-define(ST_LOCATION, ?LOCATION). +-endif. + +-define(ALERT_REC(Level,Desc), #alert{level=Level,description=Desc,where= ?ST_LOCATION}). +-define(ALERT_REC(Level,Desc,Reason), #alert{level=Level,description=Desc,where=?ST_LOCATION,reason=Reason}). -define(MAX_ALERTS, 10). diff --git a/lib/ssl/src/ssl_certificate.erl b/lib/ssl/src/ssl_certificate.erl index ebaa9a6bec..375a416b95 100644 --- a/lib/ssl/src/ssl_certificate.erl +++ b/lib/ssl/src/ssl_certificate.erl @@ -548,10 +548,10 @@ is_supported_signature_algorithm_1_2(#'OTPCertificate'{signatureAlgorithm = is_supported_signature_algorithm_1_2(#'OTPCertificate'{signatureAlgorithm = SignAlg}, SignAlgs) -> Scheme = ssl_cipher:signature_algorithm_to_scheme(SignAlg), {Hash, Sign, _ } = ssl_cipher:scheme_to_components(Scheme), - lists:member({pre_1_3_hash(Hash), pre_1_3_sign(Sign)}, SignAlgs). + ssl_cipher:is_supported_sign({pre_1_3_hash(Hash), pre_1_3_sign(Sign)}, ssl_cipher:signature_schemes_1_2(SignAlgs)). is_supported_signature_algorithm_1_3(#'OTPCertificate'{signatureAlgorithm = SignAlg}, SignAlgs) -> Scheme = ssl_cipher:signature_algorithm_to_scheme(SignAlg), - lists:member(Scheme, SignAlgs). + ssl_cipher:is_supported_sign(Scheme, SignAlgs). pre_1_3_sign(rsa_pkcs1) -> rsa; diff --git a/lib/ssl/src/ssl_cipher.erl b/lib/ssl/src/ssl_cipher.erl index 35b2da773b..52d37d9093 100644 --- a/lib/ssl/src/ssl_cipher.erl +++ b/lib/ssl/src/ssl_cipher.erl @@ -58,7 +58,9 @@ calc_mac_hash/4, calc_mac_hash/6, is_stream_ciphersuite/1, + is_supported_sign/2, signature_scheme/1, + signature_schemes_1_2/1, scheme_to_components/1, hash_size/1, effective_key_bits/1, @@ -564,6 +566,119 @@ hash_size(sha384) -> hash_size(sha512) -> 64. +is_supported_sign({Hash, rsa} = SignAlgo, HashSigns) -> %% PRE TLS-1.3 + lists:member(SignAlgo, HashSigns) orelse + lists:member({Hash, rsa_pss_rsae}, HashSigns); +is_supported_sign(rsa_pkcs1_sha256 = SignAlgo, HashSigns) -> %% TLS-1.3 leagcy + lists:member(SignAlgo, HashSigns) orelse + lists:member(rsa_pss_rsae_sha256, HashSigns); +is_supported_sign(rsa_pkcs1_sha384 = SignAlgo, HashSigns) -> %% TLS-1.3 leagcy + lists:member(SignAlgo, HashSigns) orelse + lists:member(rsa_pss_rsae_sha384, HashSigns); +is_supported_sign(rsa_pkcs1_sha512 = SignAlgo, HashSigns) -> %% TLS-1.3 leagcy + lists:member(SignAlgo, HashSigns) orelse + lists:member(rsa_pss_rsae_sha512, HashSigns); +is_supported_sign(SignAlgo, HashSigns) -> %% PRE TLS-1.3 SignAlgo::tuple() TLS-1.3 SignAlgo::atom() + lists:member(SignAlgo, HashSigns). + +signature_scheme(rsa_pkcs1_sha256) -> ?RSA_PKCS1_SHA256; +signature_scheme(rsa_pkcs1_sha384) -> ?RSA_PKCS1_SHA384; +signature_scheme(rsa_pkcs1_sha512) -> ?RSA_PKCS1_SHA512; +signature_scheme(ecdsa_secp256r1_sha256) -> ?ECDSA_SECP256R1_SHA256; +signature_scheme(ecdsa_secp384r1_sha384) -> ?ECDSA_SECP384R1_SHA384; +signature_scheme(ecdsa_secp521r1_sha512) -> ?ECDSA_SECP521R1_SHA512; +signature_scheme(rsa_pss_rsae_sha256) -> ?RSA_PSS_RSAE_SHA256; +signature_scheme(rsa_pss_rsae_sha384) -> ?RSA_PSS_RSAE_SHA384; +signature_scheme(rsa_pss_rsae_sha512) -> ?RSA_PSS_RSAE_SHA512; +signature_scheme(eddsa_ed25519) -> ?ED25519; +signature_scheme(eddsa_ed448) -> ?ED448; +signature_scheme(rsa_pss_pss_sha256) -> ?RSA_PSS_PSS_SHA256; +signature_scheme(rsa_pss_pss_sha384) -> ?RSA_PSS_PSS_SHA384; +signature_scheme(rsa_pss_pss_sha512) -> ?RSA_PSS_PSS_SHA512; +signature_scheme(rsa_pkcs1_sha1) -> ?RSA_PKCS1_SHA1; +signature_scheme(ecdsa_sha1) -> ?ECDSA_SHA1; +%% New algorithms on legacy format +signature_scheme({sha512, rsa_pss_pss}) -> + ?RSA_PSS_PSS_SHA512; +signature_scheme({sha384, rsa_pss_pss}) -> + ?RSA_PSS_PSS_SHA384; +signature_scheme({sha256, rsa_pss_pss}) -> + ?RSA_PSS_PSS_SHA256; +signature_scheme({sha512, rsa_pss_rsae}) -> + ?RSA_PSS_RSAE_SHA512; +signature_scheme({sha384, rsa_pss_rsae}) -> + ?RSA_PSS_RSAE_SHA384; +signature_scheme({sha256, rsa_pss_rsae}) -> + ?RSA_PSS_RSAE_SHA256; +%% Handling legacy signature algorithms +signature_scheme({Hash0, Sign0}) -> + Hash = hash_algorithm(Hash0), + Sign = sign_algorithm(Sign0), + <<?UINT16(SigAlg)>> = <<?BYTE(Hash),?BYTE(Sign)>>, + SigAlg; +signature_scheme(?RSA_PKCS1_SHA256) -> rsa_pkcs1_sha256; +signature_scheme(?RSA_PKCS1_SHA384) -> rsa_pkcs1_sha384; +signature_scheme(?RSA_PKCS1_SHA512) -> rsa_pkcs1_sha512; +signature_scheme(?ECDSA_SECP256R1_SHA256) -> ecdsa_secp256r1_sha256; +signature_scheme(?ECDSA_SECP384R1_SHA384) -> ecdsa_secp384r1_sha384; +signature_scheme(?ECDSA_SECP521R1_SHA512) -> ecdsa_secp521r1_sha512; +signature_scheme(?RSA_PSS_RSAE_SHA256) -> rsa_pss_rsae_sha256; +signature_scheme(?RSA_PSS_RSAE_SHA384) -> rsa_pss_rsae_sha384; +signature_scheme(?RSA_PSS_RSAE_SHA512) -> rsa_pss_rsae_sha512; +signature_scheme(?ED25519) -> eddsa_ed25519; +signature_scheme(?ED448) -> eddsa_ed448; +signature_scheme(?RSA_PSS_PSS_SHA256) -> rsa_pss_pss_sha256; +signature_scheme(?RSA_PSS_PSS_SHA384) -> rsa_pss_pss_sha384; +signature_scheme(?RSA_PSS_PSS_SHA512) -> rsa_pss_pss_sha512; +signature_scheme(?RSA_PKCS1_SHA1) -> rsa_pkcs1_sha1; +signature_scheme(?ECDSA_SHA1) -> ecdsa_sha1; +%% Handling legacy signature algorithms for logging purposes. These algorithms +%% cannot be used in TLS 1.3 handshakes. +signature_scheme(SignAlgo) when is_integer(SignAlgo) -> + <<?BYTE(Hash),?BYTE(Sign)>> = <<?UINT16(SignAlgo)>>, + try + {ssl_cipher:hash_algorithm(Hash), ssl_cipher:sign_algorithm(Sign)} + catch + _:_ -> + unassigned + end; +signature_scheme(_) -> unassigned. + +signature_schemes_1_2(SigAlgs) -> + lists:foldl(fun(Alg, Acc) when is_atom(Alg) -> + case scheme_to_components(Alg) of + {Hash, Sign = rsa_pss_pss,_} -> + [{Hash, Sign} | Acc]; + {Hash, Sign = rsa_pss_rsae,_} -> + [{Hash, Sign} | Acc]; + {_, _, _} -> + Acc + end; + (Alg, Acc) -> + [Alg| Acc] + end, [], SigAlgs). + +%% TODO: reserved code points? + +scheme_to_components(rsa_pkcs1_sha256) -> {sha256, rsa_pkcs1, undefined}; +scheme_to_components(rsa_pkcs1_sha384) -> {sha384, rsa_pkcs1, undefined}; +scheme_to_components(rsa_pkcs1_sha512) -> {sha512, rsa_pkcs1, undefined}; +scheme_to_components(ecdsa_secp256r1_sha256) -> {sha256, ecdsa, secp256r1}; +scheme_to_components(ecdsa_secp384r1_sha384) -> {sha384, ecdsa, secp384r1}; +scheme_to_components(ecdsa_secp521r1_sha512) -> {sha512, ecdsa, secp521r1}; +scheme_to_components(rsa_pss_rsae_sha256) -> {sha256, rsa_pss_rsae, undefined}; +scheme_to_components(rsa_pss_rsae_sha384) -> {sha384, rsa_pss_rsae, undefined}; +scheme_to_components(rsa_pss_rsae_sha512) -> {sha512, rsa_pss_rsae, undefined}; +scheme_to_components(eddsa_ed25519) -> {none, eddsa, ed25519}; +scheme_to_components(eddsa_ed448) -> {none, eddsa, ed448}; +scheme_to_components(rsa_pss_pss_sha256) -> {sha256, rsa_pss_pss, undefined}; +scheme_to_components(rsa_pss_pss_sha384) -> {sha384, rsa_pss_pss, undefined}; +scheme_to_components(rsa_pss_pss_sha512) -> {sha512, rsa_pss_pss, undefined}; +scheme_to_components(rsa_pkcs1_sha1) -> {sha1, rsa_pkcs1, undefined}; +scheme_to_components(ecdsa_sha1) -> {sha1, ecdsa, undefined}; +%% Handling legacy signature algorithms +scheme_to_components({Hash,Sign}) -> {Hash, Sign, undefined}. + %%-------------------------------------------------------------------- %%% Internal functions %%-------------------------------------------------------------------- @@ -752,71 +867,6 @@ sign_algorithm(Other) when is_integer(Other) andalso ((Other >= 4) and (Other =< sign_algorithm(Other) when is_integer(Other) andalso ((Other >= 224) and (Other =< 255)) -> Other. -signature_scheme(rsa_pkcs1_sha256) -> ?RSA_PKCS1_SHA256; -signature_scheme(rsa_pkcs1_sha384) -> ?RSA_PKCS1_SHA384; -signature_scheme(rsa_pkcs1_sha512) -> ?RSA_PKCS1_SHA512; -signature_scheme(ecdsa_secp256r1_sha256) -> ?ECDSA_SECP256R1_SHA256; -signature_scheme(ecdsa_secp384r1_sha384) -> ?ECDSA_SECP384R1_SHA384; -signature_scheme(ecdsa_secp521r1_sha512) -> ?ECDSA_SECP521R1_SHA512; -signature_scheme(rsa_pss_rsae_sha256) -> ?RSA_PSS_RSAE_SHA256; -signature_scheme(rsa_pss_rsae_sha384) -> ?RSA_PSS_RSAE_SHA384; -signature_scheme(rsa_pss_rsae_sha512) -> ?RSA_PSS_RSAE_SHA512; -signature_scheme(eddsa_ed25519) -> ?ED25519; -signature_scheme(eddsa_ed448) -> ?ED448; -signature_scheme(rsa_pss_pss_sha256) -> ?RSA_PSS_PSS_SHA256; -signature_scheme(rsa_pss_pss_sha384) -> ?RSA_PSS_PSS_SHA384; -signature_scheme(rsa_pss_pss_sha512) -> ?RSA_PSS_PSS_SHA512; -signature_scheme(rsa_pkcs1_sha1) -> ?RSA_PKCS1_SHA1; -signature_scheme(ecdsa_sha1) -> ?ECDSA_SHA1; -%% Handling legacy signature algorithms -signature_scheme({Hash0, Sign0}) -> - Hash = hash_algorithm(Hash0), - Sign = sign_algorithm(Sign0), - <<?UINT16(SigAlg)>> = <<?BYTE(Hash),?BYTE(Sign)>>, - SigAlg; -signature_scheme(?RSA_PKCS1_SHA256) -> rsa_pkcs1_sha256; -signature_scheme(?RSA_PKCS1_SHA384) -> rsa_pkcs1_sha384; -signature_scheme(?RSA_PKCS1_SHA512) -> rsa_pkcs1_sha512; -signature_scheme(?ECDSA_SECP256R1_SHA256) -> ecdsa_secp256r1_sha256; -signature_scheme(?ECDSA_SECP384R1_SHA384) -> ecdsa_secp384r1_sha384; -signature_scheme(?ECDSA_SECP521R1_SHA512) -> ecdsa_secp521r1_sha512; -signature_scheme(?RSA_PSS_RSAE_SHA256) -> rsa_pss_rsae_sha256; -signature_scheme(?RSA_PSS_RSAE_SHA384) -> rsa_pss_rsae_sha384; -signature_scheme(?RSA_PSS_RSAE_SHA512) -> rsa_pss_rsae_sha512; -signature_scheme(?ED25519) -> eddsa_ed25519; -signature_scheme(?ED448) -> eddsa_ed448; -signature_scheme(?RSA_PSS_PSS_SHA256) -> rsa_pss_pss_sha256; -signature_scheme(?RSA_PSS_PSS_SHA384) -> rsa_pss_pss_sha384; -signature_scheme(?RSA_PSS_PSS_SHA512) -> rsa_pss_pss_sha512; -signature_scheme(?RSA_PKCS1_SHA1) -> rsa_pkcs1_sha1; -signature_scheme(?ECDSA_SHA1) -> ecdsa_sha1; -%% Handling legacy signature algorithms for logging purposes. These algorithms -%% cannot be used in TLS 1.3 handshakes. -signature_scheme(SignAlgo) when is_integer(SignAlgo) -> - <<?BYTE(Hash),?BYTE(Sign)>> = <<?UINT16(SignAlgo)>>, - {ssl_cipher:hash_algorithm(Hash), ssl_cipher:sign_algorithm(Sign)}; -signature_scheme(_) -> unassigned. -%% TODO: reserved code points? - -scheme_to_components(rsa_pkcs1_sha256) -> {sha256, rsa_pkcs1, undefined}; -scheme_to_components(rsa_pkcs1_sha384) -> {sha384, rsa_pkcs1, undefined}; -scheme_to_components(rsa_pkcs1_sha512) -> {sha512, rsa_pkcs1, undefined}; -scheme_to_components(ecdsa_secp256r1_sha256) -> {sha256, ecdsa, secp256r1}; -scheme_to_components(ecdsa_secp384r1_sha384) -> {sha384, ecdsa, secp384r1}; -scheme_to_components(ecdsa_secp521r1_sha512) -> {sha512, ecdsa, secp521r1}; -scheme_to_components(rsa_pss_rsae_sha256) -> {sha256, rsa_pss_rsae, undefined}; -scheme_to_components(rsa_pss_rsae_sha384) -> {sha384, rsa_pss_rsae, undefined}; -scheme_to_components(rsa_pss_rsae_sha512) -> {sha512, rsa_pss_rsae, undefined}; -scheme_to_components(eddsa_ed25519) -> {none, eddsa, ed25519}; -scheme_to_components(eddsa_ed448) -> {none, eddsa, ed448}; -scheme_to_components(rsa_pss_pss_sha256) -> {sha256, rsa_pss_pss, undefined}; -scheme_to_components(rsa_pss_pss_sha384) -> {sha384, rsa_pss_pss, undefined}; -scheme_to_components(rsa_pss_pss_sha512) -> {sha512, rsa_pss_pss, undefined}; -scheme_to_components(rsa_pkcs1_sha1) -> {sha1, rsa_pkcs1, undefined}; -scheme_to_components(ecdsa_sha1) -> {sha1, ecdsa, undefined}; -%% Handling legacy signature algorithms -scheme_to_components({Hash,Sign}) -> {Hash, Sign, undefined}. - signature_algorithm_to_scheme(#'SignatureAlgorithm'{algorithm = ?'id-RSASSA-PSS', parameters = #'RSASSA-PSS-params'{ maskGenAlgorithm = @@ -842,7 +892,7 @@ signature_algorithm_to_scheme(#'SignatureAlgorithm'{algorithm = ?'ecdsa-with-SHA signature_algorithm_to_scheme(#'SignatureAlgorithm'{algorithm = ?'ecdsa-with-SHA384'}) -> ecdsa_secp384r1_sha384; signature_algorithm_to_scheme(#'SignatureAlgorithm'{algorithm = ?'ecdsa-with-SHA512'}) -> - ecdsa_secp512r1_sha512; + ecdsa_secp521r1_sha512; signature_algorithm_to_scheme(#'SignatureAlgorithm'{algorithm = ?'sha-1WithRSAEncryption'}) -> rsa_pkcs1_sha1; signature_algorithm_to_scheme(#'SignatureAlgorithm'{algorithm = ?sha1WithRSAEncryption}) -> diff --git a/lib/ssl/src/ssl_connection_sup.erl b/lib/ssl/src/ssl_connection_sup.erl index d930ecf2fa..f5d170934c 100644 --- a/lib/ssl/src/ssl_connection_sup.erl +++ b/lib/ssl/src/ssl_connection_sup.erl @@ -44,32 +44,33 @@ start_link() -> %%%========================================================================= init([]) -> - - TLSSup = tls_sup_child_spec(), - DTLSSup = dtls_sup_child_spec(), - - {ok, {{one_for_one, 10, 3600}, [TLSSup, DTLSSup]}}. + ChildSpecs = [tls_sup_child_spec(), dtls_sup_child_spec()], + SupFlags = #{strategy => one_for_one, + intensity => 10, + period => 3600 + }, + {ok, {SupFlags, ChildSpecs}}. + %%-------------------------------------------------------------------- %%% Internal functions %%-------------------------------------------------------------------- tls_sup_child_spec() -> - Name = tls_sup, - StartFunc = {tls_sup, start_link, []}, - Restart = permanent, - Shutdown = 4000, - Modules = [tls_sup], - Type = supervisor, - {Name, StartFunc, Restart, Shutdown, Type, Modules}. + #{id => tls_sup, + start => {tls_sup, start_link, []}, + restart => permanent, + shutdown => 4000, + modules => [tls_sup], + type => supervisor + }. dtls_sup_child_spec() -> - Name = dtls_sup, - StartFunc = {dtls_sup, start_link, []}, - Restart = permanent, - Shutdown = 4000, - Modules = [dtls_sup], - Type = supervisor, - {Name, StartFunc, Restart, Shutdown, Type, Modules}. - + #{id => dtls_sup, + start => {dtls_sup, start_link, []}, + restart => permanent, + shutdown => 4000, + modules => [dtls_sup], + type => supervisor + }. diff --git a/lib/ssl/src/ssl_crl.erl b/lib/ssl/src/ssl_crl.erl index d3b66df870..be44693b86 100644 --- a/lib/ssl/src/ssl_crl.erl +++ b/lib/ssl/src/ssl_crl.erl @@ -44,8 +44,8 @@ trusted_cert_and_path(CRL, issuer_not_found, CertPath, {Db, DbRef}) -> {error, unknown_ca} -> Issuer = public_key:pkix_normalize_name(public_key:pkix_crl_issuer(CRL)), IsIssuerFun = - fun({_Key, #cert{otp=ErlCertCandidate}}, Acc) -> - verify_crl_issuer(CRL, ErlCertCandidate, Issuer, Acc); + fun({_Key, CertCandidate}, Acc) -> + verify_crl_issuer(CRL, CertCandidate, Issuer, Acc); (_, Acc) -> Acc end, @@ -63,8 +63,8 @@ trusted_cert_and_path(CRL, issuer_not_found, CertPath, {Db, DbRef}) -> search_certpath(CRL, CertPath, Db, DbRef) -> Issuer = public_key:pkix_normalize_name(public_key:pkix_crl_issuer(CRL)), IsIssuerFun = - fun(ErlCertCandidate, Acc) -> - verify_crl_issuer(CRL, ErlCertCandidate, Issuer, Acc) + fun(CertCandidate, Acc) -> + verify_crl_issuer(CRL, CertCandidate, Issuer, Acc) end, case find_issuer(IsIssuerFun, certpath, CertPath) of {ok, OtpCert} -> @@ -105,13 +105,13 @@ find_issuer(IsIssuerFun, Db, _) -> Result end. -verify_crl_issuer(CRL, #cert{otp = ErlCertCandidate}, Issuer, NotIssuer) -> - TBSCert = ErlCertCandidate#'OTPCertificate'.tbsCertificate, +verify_crl_issuer(CRL, #cert{otp = OTPCertCandidate}, Issuer, NotIssuer) -> + TBSCert = OTPCertCandidate#'OTPCertificate'.tbsCertificate, case public_key:pkix_normalize_name(TBSCert#'OTPTBSCertificate'.subject) of Issuer -> - case public_key:pkix_crl_verify(CRL, ErlCertCandidate) of + case public_key:pkix_crl_verify(CRL, OTPCertCandidate) of true -> - throw({ok, ErlCertCandidate}); + throw({ok, OTPCertCandidate}); false -> NotIssuer end; diff --git a/lib/ssl/src/ssl_dist_admin_sup.erl b/lib/ssl/src/ssl_dist_admin_sup.erl index f60806c4cb..3e10643dcd 100644 --- a/lib/ssl/src/ssl_dist_admin_sup.erl +++ b/lib/ssl/src/ssl_dist_admin_sup.erl @@ -44,31 +44,31 @@ start_link() -> %%%========================================================================= init([]) -> - PEMCache = pem_cache_child_spec(), - SessionCertManager = session_and_cert_manager_child_spec(), - {ok, {{rest_for_one, 10, 3600}, [PEMCache, SessionCertManager]}}. - - + ChildSpecs = [pem_cache_child_spec(), + session_and_cert_manager_child_spec()], + SupFlags = #{strategy => rest_for_one, + intensity => 10, + period => 3600 + }, + {ok, {SupFlags, ChildSpecs}}. %%-------------------------------------------------------------------- %%% Internal functions %%-------------------------------------------------------------------- pem_cache_child_spec() -> - Name = ssl_pem_cache_dist, - StartFunc = {ssl_pem_cache, start_link_dist, [[]]}, - Restart = permanent, - Shutdown = 4000, - Modules = [ssl_pem_cache], - Type = worker, - {Name, StartFunc, Restart, Shutdown, Type, Modules}. - + #{id => ssl_pem_cache_dist, + start => {ssl_pem_cache, start_link_dist, [[]]}, + restart => permanent, + shutdown => 4000, + modules => [ssl_pem_cache], + type => worker + }. session_and_cert_manager_child_spec() -> Opts = ssl_admin_sup:manager_opts(), - Name = ssl_dist_manager, - StartFunc = {ssl_manager, start_link_dist, [Opts]}, - Restart = permanent, - Shutdown = 4000, - Modules = [ssl_manager], - Type = worker, - {Name, StartFunc, Restart, Shutdown, Type, Modules}. - + #{id => ssl_dist_manager, + start => {ssl_manager, start_link_dist, [Opts]}, + restart => permanent, + shutdown => 4000, + modules => [ssl_manager], + type => worker + }. diff --git a/lib/ssl/src/ssl_dist_connection_sup.erl b/lib/ssl/src/ssl_dist_connection_sup.erl index 441a7577be..47c467b358 100644 --- a/lib/ssl/src/ssl_dist_connection_sup.erl +++ b/lib/ssl/src/ssl_dist_connection_sup.erl @@ -43,19 +43,22 @@ start_link() -> %%% Supervisor callback %%%========================================================================= init([]) -> - TLSSup = tls_sup_child_spec(), - {ok, {{one_for_one, 10, 3600}, [TLSSup]}}. + SupFlags = #{strategy => one_for_one, + intensity => 10, + period => 3600 + }, + ChildSpecs = [tls_sup_child_spec()], + {ok, {SupFlags, ChildSpecs}}. - %%-------------------------------------------------------------------- %%% Internal functions %%-------------------------------------------------------------------- tls_sup_child_spec() -> - Name = dist_tls_sup, - StartFunc = {tls_dist_sup, start_link, []}, - Restart = permanent, - Shutdown = 4000, - Modules = [tls_dist_sup], - Type = supervisor, - {Name, StartFunc, Restart, Shutdown, Type, Modules}. + #{id => tls_dist_sup, + start => {tls_dist_sup, start_link, []}, + restart => permanent, + shutdown => 4000, + modules => [tls_dist_sup], + type => supervisor + }. diff --git a/lib/ssl/src/ssl_dist_sup.erl b/lib/ssl/src/ssl_dist_sup.erl index ae0887c3d9..74e4775413 100644 --- a/lib/ssl/src/ssl_dist_sup.erl +++ b/lib/ssl/src/ssl_dist_sup.erl @@ -58,30 +58,34 @@ start_link() -> %%%========================================================================= init([]) -> - AdminSup = ssl_admin_child_spec(), - ConnectionSup = ssl_connection_sup(), - {ok, {{one_for_all, 10, 3600}, [AdminSup, ConnectionSup]}}. + SupFlags = #{strategy => one_for_all, + intensity => 10, + period => 3600 + }, + ChildSpecs = [ssl_admin_child_spec(), + ssl_connection_sup()], + {ok, {SupFlags, ChildSpecs}}. %%-------------------------------------------------------------------- %%% Internal functions %%-------------------------------------------------------------------- ssl_admin_child_spec() -> - Name = ssl_dist_admin_sup, - StartFunc = {ssl_dist_admin_sup, start_link , []}, - Restart = permanent, - Shutdown = 4000, - Modules = [ssl_dist_admin_sup], - Type = supervisor, - {Name, StartFunc, Restart, Shutdown, Type, Modules}. - + #{id => ssl_dist_admin_sup, + start => {ssl_dist_admin_sup, start_link , []}, + restart => permanent, + shutdown => 4000, + modules => [ssl_dist_admin_sup], + type => supervisor + }. + ssl_connection_sup() -> - Name = tls_dist_sup, - StartFunc = {tls_dist_sup, start_link, []}, - Restart = permanent, - Shutdown = 4000, - Modules = [tls_dist_sup], - Type = supervisor, - {Name, StartFunc, Restart, Shutdown, Type, Modules}. + #{id => tls_dist_sup, + start => {tls_dist_sup, start_link, []}, + restart => permanent, + shutdown => 4000, + modules => [tls_dist_sup], + type => supervisor + }. consult(File) -> case erl_prim_loader:get_file(File) of diff --git a/lib/ssl/src/ssl_gen_statem.erl b/lib/ssl/src/ssl_gen_statem.erl index a68f9e0836..3bbb820ef7 100644 --- a/lib/ssl/src/ssl_gen_statem.erl +++ b/lib/ssl/src/ssl_gen_statem.erl @@ -87,7 +87,7 @@ %% Alert and close handling -export([send_alert/3, - handle_own_alert/4, + handle_own_alert/3, handle_alert/3, handle_normal_shutdown/3, handle_trusted_certs_db/1, @@ -126,9 +126,8 @@ start_link(Role, Host, Port, Socket, Options, User, CbInfo) -> -spec init(list()) -> no_return(). %% Description: Initialization %%-------------------------------------------------------------------- -init([_Role, Sender, _Host, _Port, _Socket, {#{erl_dist := ErlDist} = TLSOpts, _, _}, _User, _CbInfo] = InitArgs) -> +init([_Role, _Sender, _Host, _Port, _Socket, {#{erl_dist := ErlDist} = TLSOpts, _, _}, _User, _CbInfo] = InitArgs) -> process_flag(trap_exit, true), - link(Sender), case ErlDist of true -> process_flag(priority, max); @@ -515,8 +514,7 @@ initial_hello({call, From}, {start, Timeout}, [{{timeout, handshake}, Timeout, close}]) catch {Ref, #alert{} = Alert} -> - handle_own_alert(Alert, RequestedVersion, init, - State0#state{start_or_recv_from = From}) + handle_own_alert(Alert, init, State0#state{start_or_recv_from = From}) end; initial_hello({call, From}, {start, Timeout}, #state{static_env = #static_env{role = Role, protocol_cb = Connection}, @@ -687,7 +685,11 @@ downgrade(info, {CloseTag, Socket}, downgrade(info, Info, State) -> tls_gen_connection:handle_info(Info, ?FUNCTION_NAME, State); downgrade(Type, Event, State) -> - tls_dtls_connection:?FUNCTION_NAME(Type, Event, State). + try + tls_dtls_connection:?FUNCTION_NAME(Type, Event, State) + catch throw:#alert{} = Alert -> + handle_own_alert(Alert, ?FUNCTION_NAME, State) + end. %%==================================================================== %% Event/Msg handling @@ -705,9 +707,8 @@ handle_common_event(internal, {protocol_record, TLSorDTLSRecord}, StateName, Connection:handle_protocol_record(TLSorDTLSRecord, StateName, State); handle_common_event(timeout, hibernate, _, _) -> {keep_state_and_data, [hibernate]}; -handle_common_event(internal, #change_cipher_spec{type = <<1>>}, StateName, - #state{connection_env = #connection_env{negotiated_version = Version}} = State) -> - handle_own_alert(?ALERT_REC(?FATAL, ?HANDSHAKE_FAILURE), Version, StateName, State); +handle_common_event(internal, #change_cipher_spec{type = <<1>>}, StateName, State) -> + handle_own_alert(?ALERT_REC(?FATAL, ?HANDSHAKE_FAILURE), StateName, State); handle_common_event({timeout, handshake}, close, _StateName, #state{start_or_recv_from = StartFrom} = State) -> {stop_and_reply, {shutdown, user_timeout}, @@ -718,10 +719,11 @@ handle_common_event({timeout, recv}, timeout, StateName, #state{start_or_recv_fr handle_common_event(internal, {recv, RecvFrom}, StateName, #state{start_or_recv_from = RecvFrom}) when StateName =/= connection -> {keep_state_and_data, [postpone]}; -handle_common_event(Type, Msg, StateName, #state{connection_env = - #connection_env{negotiated_version = Version}} = State) -> +handle_common_event(internal, new_connection, StateName, State) -> + {next_state, StateName, State}; +handle_common_event(Type, Msg, StateName, State) -> Alert = ?ALERT_REC(?FATAL,?UNEXPECTED_MESSAGE, {unexpected_msg, {Type, Msg}}), - handle_own_alert(Alert, Version, StateName, State). + handle_own_alert(Alert, StateName, State). handle_call({application_data, _Data}, _, _, _) -> %% In renegotiation priorities handshake, send data when handshake is finished @@ -939,7 +941,7 @@ send_alert(Alert, connection, #state{static_env = #static_env{protocol_cb = Conn send_alert(Alert, _, #state{static_env = #static_env{protocol_cb = Connection}} = State) -> Connection:send_alert(Alert, State). -handle_own_alert(Alert0, _, StateName, +handle_own_alert(Alert0, StateName, #state{static_env = #static_env{role = Role, protocol_cb = Connection}, ssl_options = #{log_level := LogLevel}} = State) -> @@ -1188,11 +1190,12 @@ call(FsmPid, Event) -> {error, closed}; exit:{normal, _} -> {error, closed}; + exit:{shutdown,_} -> + {error, closed}; exit:{{shutdown, _},_} -> {error, closed} end. - check_hostname(_, "") -> ?ALERT_REC(?FATAL, ?UNRECOGNIZED_NAME, empty_sni); diff --git a/lib/ssl/src/ssl_handshake.erl b/lib/ssl/src/ssl_handshake.erl index 63fbf5ffff..2d5e29fb61 100644 --- a/lib/ssl/src/ssl_handshake.erl +++ b/lib/ssl/src/ssl_handshake.erl @@ -652,8 +652,8 @@ encode_extensions([#srp{username = UserName} | Rest], Acc) -> encode_extensions(Rest, <<?UINT16(?SRP_EXT), ?UINT16(Len), ?BYTE(SRPLen), UserName/binary, Acc/binary>>); encode_extensions([#hash_sign_algos{hash_sign_algos = HashSignAlgos} | Rest], Acc) -> - SignAlgoList = << <<(ssl_cipher:hash_algorithm(Hash)):8, (ssl_cipher:sign_algorithm(Sign)):8>> || - {Hash, Sign} <- HashSignAlgos >>, + SignAlgoList = << <<(ssl_cipher:signature_scheme(SignatureScheme)):16 >> || + SignatureScheme <- HashSignAlgos >>, ListLen = byte_size(SignAlgoList), Len = ListLen + 2, encode_extensions(Rest, <<?UINT16(?SIGNATURE_ALGORITHMS_EXT), @@ -988,17 +988,30 @@ available_signature_algs(undefined, _) -> available_signature_algs(SupportedHashSigns, Version) when Version >= {3, 3} -> case contains_scheme(SupportedHashSigns) of true -> - #signature_algorithms{signature_scheme_list = SupportedHashSigns}; + case Version of + {3,3} -> + #hash_sign_algos{hash_sign_algos = ssl_cipher:signature_schemes_1_2(SupportedHashSigns)}; + _ -> + #signature_algorithms{signature_scheme_list = SupportedHashSigns} + end; false -> #hash_sign_algos{hash_sign_algos = SupportedHashSigns} end; available_signature_algs(_, _) -> undefined. + available_signature_algs(undefined, SupportedHashSigns, _, Version) when Version >= {3,3} -> SupportedHashSigns; -available_signature_algs(#hash_sign_algos{hash_sign_algos = ClientHashSigns}, SupportedHashSigns, +available_signature_algs(#hash_sign_algos{hash_sign_algos = ClientHashSigns}, SupportedHashSigns0, _, Version) when Version >= {3,3} -> + SupportedHashSigns = + case (Version == {3,3}) andalso contains_scheme(SupportedHashSigns0) of + true -> + ssl_cipher:signature_schemes_1_2(SupportedHashSigns0); + false -> + SupportedHashSigns0 + end, sets:to_list(sets:intersection(sets:from_list(ClientHashSigns), sets:from_list(SupportedHashSigns))); available_signature_algs(_, _, _, _) -> @@ -1880,14 +1893,14 @@ supported_cert_type_or_empty(Algo, Type) -> end. certificate_authorities(CertDbHandle, CertDbRef) -> - Authorities = certificate_authorities_from_db(CertDbHandle, CertDbRef), + Authorities = [ Cert || #cert{otp = Cert} <- certificate_authorities_from_db(CertDbHandle, CertDbRef)], Enc = fun(#'OTPCertificate'{tbsCertificate=TBSCert}) -> OTPSubj = TBSCert#'OTPTBSCertificate'.subject, DNEncodedBin = public_key:pkix_encode('Name', OTPSubj, otp), DNEncodedLen = byte_size(DNEncodedBin), <<?UINT16(DNEncodedLen), DNEncodedBin/binary>> end, - list_to_binary([Enc(Cert) || {_, Cert} <- Authorities]). + list_to_binary([Enc(Cert) || Cert <- Authorities]). certificate_authorities_from_db(CertDbHandle, CertDbRef) when is_reference(CertDbRef) -> ConnectionCerts = fun({{Ref, _, _}, Cert}, Acc) when Ref == CertDbRef -> @@ -3278,6 +3291,15 @@ filter_hashsigns([Suite | Suites], [#{key_exchange := KeyExchange} | Algos], Has %% In this case hashsigns is not used as the kexchange is anonaymous filter_hashsigns(Suites, Algos, HashSigns, Version, [Suite| Acc]). +do_filter_hashsigns(rsa = SignAlgo, Suite, Suites, Algos, HashSigns, {3,3} = Version, Acc) -> + case (lists:keymember(SignAlgo, 2, HashSigns) orelse + lists:keymember(rsa_pss_rsae, 2, HashSigns) orelse + lists:keymember(rsa_pss_pss, 2, HashSigns)) of + true -> + filter_hashsigns(Suites, Algos, HashSigns, Version, [Suite| Acc]); + false -> + filter_hashsigns(Suites, Algos, HashSigns, Version, Acc) + end; do_filter_hashsigns(SignAlgo, Suite, Suites, Algos, HashSigns, Version, Acc) -> case lists:keymember(SignAlgo, 2, HashSigns) of true -> @@ -3390,8 +3412,7 @@ is_acceptable_cert_type(Sign, Types) -> %% signature_algorithms_cert = undefined is_supported_sign(SignAlgo, _, HashSigns, []) -> - lists:member(SignAlgo, HashSigns); - + ssl_cipher:is_supported_sign(SignAlgo, HashSigns); %% {'SignatureAlgorithm',{1,2,840,113549,1,1,11},'NULL'} is_supported_sign({Hash, Sign}, 'NULL', _, SignatureSchemes) -> Fun = fun (Scheme, Acc) -> @@ -3408,7 +3429,6 @@ is_supported_sign({Hash, Sign}, 'NULL', _, SignatureSchemes) -> Hash =:= H1) end, lists:foldl(Fun, false, SignatureSchemes); - %% TODO: Implement validation for the curve used in the signature %% RFC 3279 - 2.2.3 ECDSA Signature Algorithm %% When the ecdsa-with-SHA1 algorithm identifier appears as the diff --git a/lib/ssl/src/ssl_listen_tracker_sup.erl b/lib/ssl/src/ssl_listen_tracker_sup.erl index 6afd1c0009..998ec5fbc3 100644 --- a/lib/ssl/src/ssl_listen_tracker_sup.erl +++ b/lib/ssl/src/ssl_listen_tracker_sup.erl @@ -51,20 +51,19 @@ start_child_dist(Args) -> %%%========================================================================= %%% Supervisor callback %%%========================================================================= -init(_O) -> - RestartStrategy = simple_one_for_one, - MaxR = 0, - MaxT = 3600, - - Name = undefined, % As simple_one_for_one is used. - StartFunc = {tls_socket, start_link, []}, - Restart = temporary, % E.g. should not be restarted - Shutdown = 4000, - Modules = [tls_socket], - Type = worker, - - ChildSpec = {Name, StartFunc, Restart, Shutdown, Type, Modules}, - {ok, {{RestartStrategy, MaxR, MaxT}, [ChildSpec]}}. +init(_) -> + SupFlags = #{strategy => simple_one_for_one, + intensity => 0, + period => 3600 + }, + ChildSpecs = [#{id => undefined, + start => {tls_socket, start_link, []}, + restart => temporary, + shutdown => 4000, + modules => [tls_socket], + type => worker + }], + {ok, {SupFlags, ChildSpecs}}. tracker_name(normal) -> ?MODULE; diff --git a/lib/ssl/src/ssl_record.erl b/lib/ssl/src/ssl_record.erl index f1cf503097..86df169408 100644 --- a/lib/ssl/src/ssl_record.erl +++ b/lib/ssl/src/ssl_record.erl @@ -42,7 +42,7 @@ set_server_verify_data/3, set_max_fragment_length/2, empty_connection_state/2, - empty_connection_state/3, + empty_connection_state/4, record_protocol_role/1, step_encryption_state/1, step_encryption_state_read/1, @@ -55,6 +55,7 @@ -export([cipher/4, cipher/5, decipher/4, cipher_aead/4, cipher_aead/5, decipher_aead/5, is_correct_mac/2, nonce_seed/3]). +-define(TLS_1_3, {3, 4}). -export_type([ssl_version/0, ssl_atom_version/0, connection_states/0, connection_state/0]). @@ -465,10 +466,12 @@ nonce_seed(_,_, CipherState) -> empty_connection_state(ConnectionEnd, BeastMitigation) -> MaxEarlyDataSize = ssl_config:get_max_early_data_size(), - empty_connection_state(ConnectionEnd, BeastMitigation, MaxEarlyDataSize). + empty_connection_state(ConnectionEnd, _Version = undefined, + BeastMitigation, MaxEarlyDataSize). %% -empty_connection_state(ConnectionEnd, BeastMitigation, MaxEarlyDataSize) -> - SecParams = empty_security_params(ConnectionEnd), +empty_connection_state(ConnectionEnd, Version, + BeastMitigation, MaxEarlyDataSize) -> + SecParams = init_security_parameters(ConnectionEnd, Version), #{security_parameters => SecParams, beast_mitigation => BeastMitigation, compression_state => undefined, @@ -483,13 +486,16 @@ empty_connection_state(ConnectionEnd, BeastMitigation, MaxEarlyDataSize) -> early_data_limit => false }. -empty_security_params(ConnectionEnd = ?CLIENT) -> - #security_parameters{connection_end = ConnectionEnd, - client_random = random()}; -empty_security_params(ConnectionEnd = ?SERVER) -> - #security_parameters{connection_end = ConnectionEnd, - server_random = random()}. -random() -> +init_security_parameters(?CLIENT, Version) -> + #security_parameters{connection_end = ?CLIENT, + client_random = make_random(Version)}; +init_security_parameters(?SERVER, Version) -> + #security_parameters{connection_end = ?SERVER, + server_random = make_random(Version)}. + +make_random({_Major, _Minor} = Version) when Version >= ?TLS_1_3 -> + ssl_cipher:random_bytes(32); +make_random(_Version) -> Secs_since_1970 = calendar:datetime_to_gregorian_seconds( calendar:universal_time()) - 62167219200, Random_28_bytes = ssl_cipher:random_bytes(28), diff --git a/lib/ssl/src/ssl_server_session_cache_sup.erl b/lib/ssl/src/ssl_server_session_cache_sup.erl index 88f068a319..80868215d8 100644 --- a/lib/ssl/src/ssl_server_session_cache_sup.erl +++ b/lib/ssl/src/ssl_server_session_cache_sup.erl @@ -44,22 +44,19 @@ start_link() -> start_child(Listner) -> supervisor:start_child(?MODULE, [Listner | [ssl_config:pre_1_3_session_opts(server)]]). - %%%========================================================================= %%% Supervisor callback %%%========================================================================= -init(_O) -> - RestartStrategy = simple_one_for_one, - MaxR = 3, - MaxT = 3600, - - Name = undefined, % As simple_one_for_one is used. - StartFunc = {ssl_server_session_cache, start_link, []}, - Restart = transient, % Should be restarted only on abnormal termination - Shutdown = 4000, - Modules = [ssl_server_session_cache], - Type = worker, - - ChildSpec = {Name, StartFunc, Restart, Shutdown, Type, Modules}, - {ok, {{RestartStrategy, MaxR, MaxT}, [ChildSpec]}}. - +init(_) -> + SupFlags = #{strategy => simple_one_for_one, + intensity => 3, + period => 3600 + }, + ChildSpecs = [#{id => undefined, + start => {ssl_server_session_cache, start_link, []}, + restart => transient, + shutdown => 4000, + modules => [ssl_server_session_cache], + type => worker + }], + {ok, {SupFlags, ChildSpecs}}. diff --git a/lib/ssl/src/ssl_sup.erl b/lib/ssl/src/ssl_sup.erl index 05a7aaaa82..1c38eae433 100644 --- a/lib/ssl/src/ssl_sup.erl +++ b/lib/ssl/src/ssl_sup.erl @@ -43,29 +43,32 @@ start_link() -> %%% Supervisor callback %%%========================================================================= -init([]) -> - {ok, {{rest_for_one, 10, 3600}, [ssl_admin_child_spec(), - ssl_connection_sup() - ]}}. +init([]) -> + SupFlags = #{strategy => rest_for_one, + intensity => 10, + period => 3600 + }, + ChildSpecs = [ssl_admin_child_spec(), + ssl_connection_sup()], + {ok, {SupFlags, ChildSpecs}}. %%-------------------------------------------------------------------- %%% Internal functions %%-------------------------------------------------------------------- ssl_admin_child_spec() -> - Name = ssl_admin_sup, - StartFunc = {ssl_admin_sup, start_link, []}, - Restart = permanent, - Shutdown = 4000, - Modules = [ssl_admin_sup], - Type = supervisor, - {Name, StartFunc, Restart, Shutdown, Type, Modules}. - + #{id => ssl_admin_sup, + start => {ssl_admin_sup, start_link, []}, + restart => permanent, + shutdown => 4000, + modules => [ssl_admin_sup], + type => supervisor + }. + ssl_connection_sup() -> - Name = ssl_connection_sup, - StartFunc = {ssl_connection_sup, start_link, []}, - Restart = permanent, - Shutdown = 4000, - Modules = [ssl_connection_sup], - Type = supervisor, - {Name, StartFunc, Restart, Shutdown, Type, Modules}. - + #{id => ssl_connection_sup, + start => {ssl_connection_sup, start_link, []}, + restart => permanent, + shutdown => 4000, + modules => [ssl_connection_sup], + type => supervisor + }. diff --git a/lib/ssl/src/ssl_upgrade_server_session_cache_sup.erl b/lib/ssl/src/ssl_upgrade_server_session_cache_sup.erl index 69169cca0d..62f1e8e4f1 100644 --- a/lib/ssl/src/ssl_upgrade_server_session_cache_sup.erl +++ b/lib/ssl/src/ssl_upgrade_server_session_cache_sup.erl @@ -69,20 +69,19 @@ start_child(Type) -> %%%========================================================================= %%% Supervisor callback %%%========================================================================= -init(_O) -> - RestartStrategy = simple_one_for_one, - MaxR = 3, - MaxT = 3600, - - Name = undefined, % As simple_one_for_one is used. - StartFunc = {ssl_server_session_cache, start_link, []}, - Restart = transient, % Should be restarted only on abnormal termination - Shutdown = 4000, - Modules = [ssl_server_session_cache], - Type = worker, - - ChildSpec = {Name, StartFunc, Restart, Shutdown, Type, Modules}, - {ok, {{RestartStrategy, MaxR, MaxT}, [ChildSpec]}}. +init(_) -> + SupFlags = #{strategy => simple_one_for_one, + intensity => 3, + period => 3600 + }, + ChildSpecs = [#{id => undefined, + start => {ssl_server_session_cache, start_link, []}, + restart => transient, + shutdown => 4000, + modules => [ssl_server_session_cache], + type => worker + }], + {ok, {SupFlags, ChildSpecs}}. sup_name(normal) -> ?MODULE; diff --git a/lib/ssl/src/tls_connection.erl b/lib/ssl/src/tls_connection.erl index ff06b5dc71..7f0be2dbc5 100644 --- a/lib/ssl/src/tls_connection.erl +++ b/lib/ssl/src/tls_connection.erl @@ -215,7 +215,7 @@ hello(internal, #client_hello{extensions = Extensions} = Hello, [{reply, From, {ok, Extensions}}]}; hello(internal, #server_hello{extensions = Extensions} = Hello, #state{ssl_options = #{ - handshake := hello}, + handshake := hello}, handshake_env = HsEnv, start_or_recv_from = From} = State) -> {next_state, user_hello, @@ -229,52 +229,56 @@ hello(internal, #client_hello{client_version = ClientVersion} = Hello, #state{ss %% Continue in TLS 1.3 'start' state {next_state, start, State0, [{change_callback_module, tls_connection_1_3}, {next_event, internal, Hello}]}; tls_1_0_to_1_2_fsm -> - case handle_client_hello(Hello, State0) of + try handle_client_hello(Hello, State0) of {ServerHelloExt, Type, State} -> - {next_state, hello, State, [{next_event, internal, {common_client_hello, Type, ServerHelloExt}}]}; - Alert -> - ssl_gen_statem:handle_own_alert(Alert, ClientVersion, hello, - State0#state{connection_env = CEnv#connection_env{negotiated_version - = ClientVersion}}) + {next_state, hello, State, [{next_event, internal, {common_client_hello, Type, ServerHelloExt}}]} + catch throw:#alert{} = Alert -> + State = State0#state{connection_env = CEnv#connection_env{negotiated_version = ClientVersion}}, + ssl_gen_statem:handle_own_alert(Alert, hello, State) end end; hello(internal, #server_hello{} = Hello, #state{connection_states = ConnectionStates0, - connection_env = #connection_env{negotiated_version = ReqVersion} = CEnv, + connection_env = CEnv, static_env = #static_env{role = client}, handshake_env = #handshake_env{ ocsp_stapling_state = OcspState0, renegotiation = {Renegotiation, _}} = HsEnv, session = #session{session_id = OldId}, - ssl_options = SslOptions} = State) -> - case tls_handshake:hello(Hello, SslOptions, ConnectionStates0, Renegotiation, OldId) of - #alert{} = Alert -> - ssl_gen_statem:handle_own_alert(Alert, ReqVersion, hello, - State#state{connection_env = - CEnv#connection_env{negotiated_version = ReqVersion} - }); - %% Legacy TLS 1.2 and older - {Version, NewId, ConnectionStates, ProtoExt, Protocol, OcspState} -> - tls_dtls_connection:handle_session(Hello, - Version, NewId, ConnectionStates, ProtoExt, Protocol, - State#state{ - handshake_env = HsEnv#handshake_env{ - ocsp_stapling_state = maps:merge(OcspState0,OcspState)}}); - %% TLS 1.3 - {next_state, wait_sh, SelectedVersion, OcspState} -> - %% Continue in TLS 1.3 'wait_sh' state - {next_state, wait_sh, - State#state{handshake_env = HsEnv#handshake_env{ocsp_stapling_state = maps:merge(OcspState0,OcspState)}, - connection_env = CEnv#connection_env{negotiated_version = SelectedVersion}}, - [{change_callback_module, tls_connection_1_3}, {next_event, internal, Hello}]} + ssl_options = SslOptions} = State) -> + try + case tls_handshake:hello(Hello, SslOptions, ConnectionStates0, Renegotiation, OldId) of + %% Legacy TLS 1.2 and older + {Version, NewId, ConnectionStates, ProtoExt, Protocol, OcspState} -> + tls_dtls_connection:handle_session(Hello, + Version, NewId, ConnectionStates, ProtoExt, Protocol, + State#state{ + handshake_env = HsEnv#handshake_env{ + ocsp_stapling_state = maps:merge(OcspState0,OcspState)}}); + %% TLS 1.3 + {next_state, wait_sh, SelectedVersion, OcspState} -> + %% Continue in TLS 1.3 'wait_sh' state + {next_state, wait_sh, + State#state{handshake_env = HsEnv#handshake_env{ocsp_stapling_state = maps:merge(OcspState0,OcspState)}, + connection_env = CEnv#connection_env{negotiated_version = SelectedVersion}}, + [{change_callback_module, tls_connection_1_3}, {next_event, internal, Hello}]} + end + catch throw:#alert{} = Alert -> + ssl_gen_statem:handle_own_alert(Alert, hello, State) end; hello(info, Event, State) -> tls_gen_connection:handle_info(Event, ?FUNCTION_NAME, State); hello(Type, Event, State) -> - tls_dtls_connection:gen_handshake(?FUNCTION_NAME, Type, Event, State). + try tls_dtls_connection:gen_handshake(?FUNCTION_NAME, Type, Event, State) + catch throw:#alert{} = Alert -> + ssl_gen_statem:handle_own_alert(Alert, ?FUNCTION_NAME, State) + end. user_hello(Type, Event, State) -> - tls_dtls_connection:gen_handshake(?FUNCTION_NAME, Type, Event, State). + try tls_dtls_connection:gen_handshake(?FUNCTION_NAME, Type, Event, State) + catch throw:#alert{} = Alert -> + ssl_gen_statem:handle_own_alert(Alert, ?FUNCTION_NAME, State) + end. %%-------------------------------------------------------------------- -spec abbreviated(gen_statem:event_type(), term(), #state{}) -> @@ -283,7 +287,10 @@ user_hello(Type, Event, State) -> abbreviated(info, Event, State) -> gen_info(Event, ?FUNCTION_NAME, State); abbreviated(Type, Event, State) -> - tls_dtls_connection:gen_handshake(?FUNCTION_NAME, Type, Event, State). + try tls_dtls_connection:gen_handshake(?FUNCTION_NAME, Type, Event, State) + catch throw:#alert{} = Alert -> + ssl_gen_statem:handle_own_alert(Alert, ?FUNCTION_NAME, State) + end. %%-------------------------------------------------------------------- -spec wait_ocsp_stapling(gen_statem:event_type(), term(), #state{}) -> @@ -292,7 +299,10 @@ abbreviated(Type, Event, State) -> wait_ocsp_stapling(info, Event, State) -> gen_info(Event, ?FUNCTION_NAME, State); wait_ocsp_stapling(Type, Event, State) -> - tls_dtls_connection:gen_handshake(?FUNCTION_NAME, Type, Event, State). + try tls_dtls_connection:gen_handshake(?FUNCTION_NAME, Type, Event, State) + catch throw:#alert{} = Alert -> + ssl_gen_statem:handle_own_alert(Alert, ?FUNCTION_NAME, State) + end. %%-------------------------------------------------------------------- -spec certify(gen_statem:event_type(), term(), #state{}) -> @@ -301,7 +311,10 @@ wait_ocsp_stapling(Type, Event, State) -> certify(info, Event, State) -> gen_info(Event, ?FUNCTION_NAME, State); certify(Type, Event, State) -> - tls_dtls_connection:gen_handshake(?FUNCTION_NAME, Type, Event, State). + try tls_dtls_connection:gen_handshake(?FUNCTION_NAME, Type, Event, State) + catch throw:#alert{} = Alert -> + ssl_gen_statem:handle_own_alert(Alert, ?FUNCTION_NAME, State) + end. %%-------------------------------------------------------------------- -spec cipher(gen_statem:event_type(), term(), #state{}) -> @@ -310,7 +323,10 @@ certify(Type, Event, State) -> cipher(info, Event, State) -> gen_info(Event, ?FUNCTION_NAME, State); cipher(Type, Event, State) -> - tls_dtls_connection:gen_handshake(?FUNCTION_NAME, Type, Event, State). + try tls_dtls_connection:gen_handshake(?FUNCTION_NAME, Type, Event, State) + catch throw:#alert{} = Alert -> + ssl_gen_statem:handle_own_alert(Alert, ?FUNCTION_NAME, State) + end. %%-------------------------------------------------------------------- -spec connection(gen_statem:event_type(), @@ -395,7 +411,10 @@ connection(internal, #client_hello{}, State = tls_gen_connection:reinit_handshake_data(State0), tls_gen_connection:next_event(?FUNCTION_NAME, no_record, State); connection(Type, Event, State) -> - tls_dtls_connection:?FUNCTION_NAME(Type, Event, State). + try tls_dtls_connection:?FUNCTION_NAME(Type, Event, State) + catch throw:#alert{} = Alert -> + ssl_gen_statem:handle_own_alert(Alert, ?FUNCTION_NAME, State) + end. %%-------------------------------------------------------------------- -spec downgrade(gen_statem:event_type(), term(), #state{}) -> @@ -417,8 +436,7 @@ terminate({shutdown, {sender_died, Reason}}, _StateName, ssl_gen_statem:handle_trusted_certs_db(State), tls_gen_connection:close(Reason, Socket, Transport, undefined, undefined); terminate(Reason, StateName, State) -> - catch ssl_gen_statem:terminate(Reason, StateName, State), - ensure_sender_terminate(Reason, State). + ssl_gen_statem:terminate(Reason, StateName, State). format_status(Type, Data) -> ssl_gen_statem:format_status(Type, Data). @@ -433,8 +451,12 @@ initial_state(Role, Sender, Host, Port, Socket, {SSLOptions, SocketOptions, Trac {CbModule, DataTag, CloseTag, ErrorTag, PassiveTag}) -> #{beast_mitigation := BeastMitigation, erl_dist := IsErlDist, + %% Use highest supported version for client/server random nonce generation + versions := [Version|_], client_renegotiation := ClientRenegotiation} = SSLOptions, - ConnectionStates = tls_record:init_connection_states(Role, BeastMitigation), + ConnectionStates = tls_record:init_connection_states(Role, + Version, + BeastMitigation), #{session_cb := SessionCacheCb} = ssl_config:pre_1_3_session_opts(Role), UserMonitor = erlang:monitor(process, User), InitStatEnv = #static_env{ @@ -474,86 +496,65 @@ initial_state(Role, Sender, Host, Port, Socket, {SSLOptions, SocketOptions, Trac }. handle_client_hello(#client_hello{client_version = ClientVersion} = Hello, State0) -> - case tls_dtls_connection:handle_sni_extension(State0, Hello) of - #state{connection_states = ConnectionStates0, - static_env = #static_env{trackers = Trackers}, - handshake_env = #handshake_env{ - kex_algorithm = KeyExAlg, - renegotiation = {Renegotiation, _}, - negotiated_protocol = CurrentProtocol, - sni_guided_cert_selection = SNICertSelection} = HsEnv, - connection_env = CEnv, - session = #session{own_certificates = OwnCerts} = Session0, - ssl_options = SslOpts} = State -> - SessionTracker = proplists:get_value(session_id_tracker, Trackers), - case tls_handshake:hello(Hello, - SslOpts, - {SessionTracker, Session0, - ConnectionStates0, OwnCerts, KeyExAlg}, - Renegotiation) of - #alert{} = Alert -> - Alert; - {Version, {Type, Session}, - ConnectionStates, Protocol0, ServerHelloExt0, HashSign} -> - Protocol = case Protocol0 of - undefined -> CurrentProtocol; - _ -> Protocol0 - end, - ServerHelloExt = - case SNICertSelection of - true -> - ServerHelloExt0#{sni => #sni{hostname = ""}}; - false -> - ServerHelloExt0 - end, - {ServerHelloExt, Type, State#state{connection_states = ConnectionStates, - connection_env = CEnv#connection_env{negotiated_version = Version}, - handshake_env = HsEnv#handshake_env{ - hashsign_algorithm = HashSign, - client_hello_version = ClientVersion, - negotiated_protocol = Protocol}, - session = Session - }} - end; - #alert{} = Alert -> - Alert - end. + State = tls_dtls_connection:handle_sni_extension(State0, Hello), + #state{connection_states = ConnectionStates0, + static_env = #static_env{trackers = Trackers}, + handshake_env = #handshake_env{ + kex_algorithm = KeyExAlg, + renegotiation = {Renegotiation, _}, + negotiated_protocol = CurrentProtocol, + sni_guided_cert_selection = SNICertSelection} = HsEnv, + connection_env = CEnv, + session = #session{own_certificates = OwnCerts} = Session0, + ssl_options = SslOpts} = State, + SessionTracker = proplists:get_value(session_id_tracker, Trackers), + {Version, {Type, Session}, + ConnectionStates, Protocol0, ServerHelloExt0, HashSign} = + tls_handshake:hello(Hello, + SslOpts, + {SessionTracker, Session0, + ConnectionStates0, OwnCerts, KeyExAlg}, + Renegotiation), + Protocol = case Protocol0 of + undefined -> CurrentProtocol; + _ -> Protocol0 + end, + ServerHelloExt = + case SNICertSelection of + true -> + ServerHelloExt0#{sni => #sni{hostname = ""}}; + false -> + ServerHelloExt0 + end, + {ServerHelloExt, Type, State#state{connection_states = ConnectionStates, + connection_env = CEnv#connection_env{negotiated_version = Version}, + handshake_env = HsEnv#handshake_env{ + hashsign_algorithm = HashSign, + client_hello_version = ClientVersion, + negotiated_protocol = Protocol}, + session = Session + }}. -gen_info(Event, connection = StateName, #state{connection_env = #connection_env{negotiated_version = Version}} = State) -> - try tls_gen_connection:handle_info(Event, StateName, State) of - Result -> - Result - catch +gen_info(Event, connection = StateName, State) -> + try + tls_gen_connection:handle_info(Event, StateName, State) + catch _:_ -> ssl_gen_statem:handle_own_alert(?ALERT_REC(?FATAL, ?INTERNAL_ERROR, - malformed_data), - Version, StateName, State) + malformed_data), + StateName, State) end; -gen_info(Event, StateName, #state{connection_env = #connection_env{negotiated_version = Version}} = State) -> - try tls_gen_connection:handle_info(Event, StateName, State) of - Result -> - Result - catch +gen_info(Event, StateName, State) -> + try + tls_gen_connection:handle_info(Event, StateName, State) + catch _:_ -> ssl_gen_statem:handle_own_alert(?ALERT_REC(?FATAL, ?HANDSHAKE_FAILURE, - malformed_handshake_data), - Version, StateName, State) + malformed_handshake_data), + StateName, State) end. - -ensure_sender_terminate(downgrade, _) -> - ok; %% Do not terminate sender during downgrade phase -ensure_sender_terminate(_, #state{protocol_specific = #{sender := Sender}}) -> - %% Make sure TLS sender dies when connection process is terminated normally - %% This is needed if the tls_sender is blocked in prim_inet:send - Kill = fun() -> - receive - after 5000 -> - catch (exit(Sender, kill)) - end - end, - spawn(Kill). choose_tls_fsm(#{versions := Versions}, #client_hello{ diff --git a/lib/ssl/src/tls_connection_1_3.erl b/lib/ssl/src/tls_connection_1_3.erl index 2d123bab43..bacdf791ad 100644 --- a/lib/ssl/src/tls_connection_1_3.erl +++ b/lib/ssl/src/tls_connection_1_3.erl @@ -232,11 +232,10 @@ config_error(Type, Event, State) -> ssl_gen_statem:?FUNCTION_NAME(Type, Event, State). -user_hello({call, From}, cancel, #state{connection_env = #connection_env{negotiated_version = Version}} - = State) -> +user_hello({call, From}, cancel, State) -> gen_statem:reply(From, ok), ssl_gen_statem:handle_own_alert(?ALERT_REC(?FATAL, ?USER_CANCELED, user_canceled), - Version, ?FUNCTION_NAME, State); + ?FUNCTION_NAME, State); user_hello({call, From}, {handshake_continue, NewOptions, Timeout}, #state{static_env = #static_env{role = Role}, handshake_env = #handshake_env{hello = Hello}, @@ -269,7 +268,7 @@ start(internal, #client_hello{extensions = Extensions} = Hello, start(internal, #client_hello{} = Hello, State0) -> case tls_handshake_1_3:do_start(Hello, State0) of #alert{} = Alert -> - ssl_gen_statem:handle_own_alert(Alert, {3,4}, start, State0); + ssl_gen_statem:handle_own_alert(Alert, start, State0); {State, start} -> {next_state, start, State, []}; {State, negotiated} -> @@ -289,7 +288,7 @@ start(internal, #server_hello{extensions = Extensions} = ServerHello, start(internal, #server_hello{} = ServerHello, State0) -> case tls_handshake_1_3:do_start(ServerHello, State0) of #alert{} = Alert -> - ssl_gen_statem:handle_own_alert(Alert, {3,4}, start, State0); + ssl_gen_statem:handle_own_alert(Alert, start, State0); {State, NextState} -> {next_state, NextState, State, []} end; @@ -303,7 +302,7 @@ negotiated(internal, #change_cipher_spec{}, State) -> negotiated(internal, Message, State0) -> case tls_handshake_1_3:do_negotiated(Message, State0) of #alert{} = Alert -> - ssl_gen_statem:handle_own_alert(Alert, {3,4}, negotiated, State0); + ssl_gen_statem:handle_own_alert(Alert, negotiated, State0); {State, NextState} -> {next_state, NextState, State, []} end; @@ -316,7 +315,7 @@ wait_cert(internal, #certificate_1_3{} = Certificate, State0) -> case tls_handshake_1_3:do_wait_cert(Certificate, State0) of {#alert{} = Alert, State} -> - ssl_gen_statem:handle_own_alert(Alert, {3,4}, wait_cert, State); + ssl_gen_statem:handle_own_alert(Alert, wait_cert, State); {State, NextState} -> tls_gen_connection:next_event(NextState, no_record, State) end; @@ -331,7 +330,7 @@ wait_cv(internal, #certificate_verify_1_3{} = CertificateVerify, State0) -> case tls_handshake_1_3:do_wait_cv(CertificateVerify, State0) of {#alert{} = Alert, State} -> - ssl_gen_statem:handle_own_alert(Alert, {3,4}, wait_cv, State); + ssl_gen_statem:handle_own_alert(Alert, wait_cv, State); {State, NextState} -> tls_gen_connection:next_event(NextState, no_record, State) end; @@ -346,7 +345,7 @@ wait_finished(internal, #finished{} = Finished, State0) -> case tls_handshake_1_3:do_wait_finished(Finished, State0) of #alert{} = Alert -> - ssl_gen_statem:handle_own_alert(Alert, {3,4}, finished, State0); + ssl_gen_statem:handle_own_alert(Alert, finished, State0); State1 -> {Record, State} = ssl_gen_statem:prepare_connection(State1, tls_gen_connection), tls_gen_connection:next_event(connection, Record, State, @@ -370,7 +369,7 @@ wait_sh(internal, #server_hello{extensions = Extensions} = Hello, #state{ssl_op wait_sh(internal, #server_hello{} = Hello, State0) -> case tls_handshake_1_3:do_wait_sh(Hello, State0) of #alert{} = Alert -> - ssl_gen_statem:handle_own_alert(Alert, {3,4}, wait_sh, State0); + ssl_gen_statem:handle_own_alert(Alert, wait_sh, State0); {State1, start, ServerHello} -> %% hello_retry_request: go to start {next_state, start, State1, [{next_event, internal, ServerHello}]}; @@ -388,7 +387,7 @@ wait_ee(internal, #change_cipher_spec{}, State) -> wait_ee(internal, #encrypted_extensions{} = EE, State0) -> case tls_handshake_1_3:do_wait_ee(EE, State0) of #alert{} = Alert -> - ssl_gen_statem:handle_own_alert(Alert, {3,4}, wait_ee, State0); + ssl_gen_statem:handle_own_alert(Alert, wait_ee, State0); {State1, NextState} -> tls_gen_connection:next_event(NextState, no_record, State1) end; @@ -403,14 +402,14 @@ wait_cert_cr(internal, #change_cipher_spec{}, State) -> wait_cert_cr(internal, #certificate_1_3{} = Certificate, State0) -> case tls_handshake_1_3:do_wait_cert_cr(Certificate, State0) of {#alert{} = Alert, State} -> - ssl_gen_statem:handle_own_alert(Alert, {3,4}, wait_cert_cr, State); + ssl_gen_statem:handle_own_alert(Alert, wait_cert_cr, State); {State1, NextState} -> tls_gen_connection:next_event(NextState, no_record, State1) end; wait_cert_cr(internal, #certificate_request_1_3{} = CertificateRequest, State0) -> case tls_handshake_1_3:do_wait_cert_cr(CertificateRequest, State0) of #alert{} = Alert -> - ssl_gen_statem:handle_own_alert(Alert, {3,4}, wait_cert_cr, State0); + ssl_gen_statem:handle_own_alert(Alert, wait_cert_cr, State0); {State1, NextState} -> tls_gen_connection:next_event(NextState, no_record, State1) end; @@ -424,7 +423,7 @@ wait_eoed(internal, #change_cipher_spec{}, State) -> wait_eoed(internal, #end_of_early_data{} = EOED, State0) -> case tls_handshake_1_3:do_wait_eoed(EOED, State0) of {#alert{} = Alert, State} -> - ssl_gen_statem:handle_own_alert(Alert, {3,4}, wait_eoed, State); + ssl_gen_statem:handle_own_alert(Alert, wait_eoed, State); {State1, NextState} -> tls_gen_connection:next_event(NextState, no_record, State1) end; @@ -442,7 +441,7 @@ connection(internal, #key_update{} = KeyUpdate, State0) -> {ok, State} -> tls_gen_connection:next_event(?FUNCTION_NAME, no_record, State); {error, State, Alert} -> - ssl_gen_statem:handle_own_alert(Alert, {3,4}, connection, State), + ssl_gen_statem:handle_own_alert(Alert, connection, State), tls_gen_connection:next_event(?FUNCTION_NAME, no_record, State) end; connection({call, From}, negotiated_protocol, @@ -468,9 +467,14 @@ downgrade(Type, Event, State) -> initial_state(Role, Sender, Host, Port, Socket, {SSLOptions, SocketOptions, Trackers}, User, {CbModule, DataTag, CloseTag, ErrorTag, PassiveTag}) -> #{erl_dist := IsErlDist, + %% Use highest supported version for client/server random nonce generation + versions := [Version|_], client_renegotiation := ClientRenegotiation} = SSLOptions, MaxEarlyDataSize = init_max_early_data_size(Role), - ConnectionStates = tls_record:init_connection_states(Role, disabled, MaxEarlyDataSize), + ConnectionStates = tls_record:init_connection_states(Role, + Version, + disabled, + MaxEarlyDataSize), UserMonitor = erlang:monitor(process, User), InitStatEnv = #static_env{ role = Role, diff --git a/lib/ssl/src/tls_connection_sup.erl b/lib/ssl/src/tls_connection_sup.erl index b7f80ad524..435639cf68 100644 --- a/lib/ssl/src/tls_connection_sup.erl +++ b/lib/ssl/src/tls_connection_sup.erl @@ -44,24 +44,22 @@ start_link_dist() -> start_child(Args) -> supervisor:start_child(?MODULE, Args). - + start_child_dist(Args) -> supervisor:start_child(tls_dist_connection_sup, Args). %%%========================================================================= %%% Supervisor callback %%%========================================================================= -init(_O) -> - RestartStrategy = simple_one_for_one, - MaxR = 0, - MaxT = 3600, - - Name = undefined, % As simple_one_for_one is used. - StartFunc = {ssl_gen_statem, start_link, []}, - Restart = temporary, % E.g. should not be restarted - Shutdown = 4000, - Modules = [ssl_gen_statem, tls_connection, tls_connection_1_3], - Type = worker, - - ChildSpec = {Name, StartFunc, Restart, Shutdown, Type, Modules}, - {ok, {{RestartStrategy, MaxR, MaxT}, [ChildSpec]}}. +init(_) -> + SupFlags = #{strategy => simple_one_for_one, + intensity => 0, + period => 3600 + }, + ChildSpecs = [#{id => undefined, + restart => temporary, + type => supervisor, + start => {tls_dyn_connection_sup, start_link, []}}], + {ok, {SupFlags, ChildSpecs}}. + + diff --git a/lib/ssl/src/tls_dist_server_sup.erl b/lib/ssl/src/tls_dist_server_sup.erl index 96603a7495..9560d05158 100644 --- a/lib/ssl/src/tls_dist_server_sup.erl +++ b/lib/ssl/src/tls_dist_server_sup.erl @@ -43,16 +43,16 @@ start_link() -> %%% Supervisor callback %%%========================================================================= -init([]) -> - ListenTracker = listen_options_tracker_child_spec(), - SessionTracker = tls_server_session_child_spec(), - Pre_1_3SessionTracker = ssl_server_session_child_spec(), - - {ok, {{one_for_all, 10, 3600}, [ListenTracker, - SessionTracker, - Pre_1_3SessionTracker - ]}}. - +init([]) -> + SupFlags = #{strategy => one_for_all, + intensity => 10, + period => 3600 + }, + ChildSpecs = [listen_options_tracker_child_spec(), + tls_server_session_child_spec(), + ssl_server_session_child_spec()], + {ok, {SupFlags, ChildSpecs}}. + %%-------------------------------------------------------------------- %%% Internal functions @@ -61,29 +61,28 @@ init([]) -> %% Handles emulated options so that they inherited by the accept %% socket, even when setopts is performed on the listen socket listen_options_tracker_child_spec() -> - Name = dist_tls_socket, - StartFunc = {ssl_listen_tracker_sup, start_link_dist, []}, - Restart = permanent, - Shutdown = 4000, - Modules = [ssl_listen_tracker_sup], - Type = supervisor, - {Name, StartFunc, Restart, Shutdown, Type, Modules}. + #{id => dist_ssl_listen_tracker_sup, + start => {ssl_listen_tracker_sup, start_link_dist, []}, + restart => permanent, + shutdown => 4000, + modules => [ssl_listen_tracker_sup], + type => supervisor + }. tls_server_session_child_spec() -> - Name = dist_tls_server_session_ticket, - StartFunc = {tls_server_session_ticket_sup, start_link_dist, []}, - Restart = permanent, - Shutdown = 4000, - Modules = [tls_server_session_ticket_sup], - Type = supervisor, - {Name, StartFunc, Restart, Shutdown, Type, Modules}. + #{id => dist_tls_server_session_ticket, + start => {tls_server_session_ticket_sup, start_link_dist, []}, + restart => permanent, + shutdown => 4000, + modules => [tls_server_session_ticket_sup], + type => supervisor + }. ssl_server_session_child_spec() -> - Name = dist_ssl_server_session_cache_sup, - StartFunc = {ssl_upgrade_server_session_cache_sup, start_link_dist, []}, - Restart = permanent, - Shutdown = 4000, - Modules = [ssl_server_session_cache_sup], - Type = supervisor, - {Name, StartFunc, Restart, Shutdown, Type, Modules}. - + #{id => dist_ssl_upgrade_server_session_cache_sup, + start => {ssl_upgrade_server_session_cache_sup, start_link_dist, []}, + restart => permanent, + shutdown => 4000, + modules => [ssl_upgrade_server_session_cache_sup], + type => supervisor + }. diff --git a/lib/ssl/src/tls_dist_sup.erl b/lib/ssl/src/tls_dist_sup.erl index 54e0a6a514..ebff741f9b 100644 --- a/lib/ssl/src/tls_dist_sup.erl +++ b/lib/ssl/src/tls_dist_sup.erl @@ -44,32 +44,34 @@ start_link() -> %%%========================================================================= init([]) -> - - TLSConnetionSup = tls_connection_child_spec(), - ServerInstanceSup = server_instance_child_spec(), - - {ok, {{one_for_one, 10, 3600}, [TLSConnetionSup, - ServerInstanceSup - ]}}. + SupFlags = #{strategy => one_for_one, + intensity => 10, + period => 3600 + }, + ChildSpecs = [tls_connection_child_spec(), + server_instance_child_spec() + ], + {ok, {SupFlags, ChildSpecs}}. %%-------------------------------------------------------------------- %%% Internal functions %%-------------------------------------------------------------------- tls_connection_child_spec() -> - Name = dist_tls_connection, - StartFunc = {tls_connection_sup, start_link_dist, []}, - Restart = permanent, - Shutdown = 4000, - Modules = [tls_connection_sup], - Type = supervisor, - {Name, StartFunc, Restart, Shutdown, Type, Modules}. - + #{id => dist_tls_connection_sup, + start => {tls_connection_sup, start_link_dist, []}, + restart => permanent, + shutdown => 4000, + modules => [tls_connection_sup], + type => supervisor + }. + server_instance_child_spec() -> - Name = dist_tls_server_sup, - StartFunc = {tls_dist_server_sup, start_link, []}, - Restart = permanent, - Shutdown = 4000, - Modules = [tls_dist_server_sup], - Type = supervisor, - {Name, StartFunc, Restart, Shutdown, Type, Modules}. + #{id => tls_dist_server_sup, + start => {tls_dist_server_sup, start_link, []}, + restart => permanent, + shutdown => 4000, + modules => [tls_dist_server_sup], + type => supervisor + }. + diff --git a/lib/ssl/src/tls_dtls_connection.erl b/lib/ssl/src/tls_dtls_connection.erl index bb138b035b..5ee5c41b44 100644 --- a/lib/ssl/src/tls_dtls_connection.erl +++ b/lib/ssl/src/tls_dtls_connection.erl @@ -22,6 +22,8 @@ %%---------------------------------------------------------------------- %% Purpose: Common handling of a TLS/SSL/DTLS connection, see also %% tls_connection.erl and dtls_connection.erl +%% +%% NOTE: All alerts are thrown out of this module %%---------------------------------------------------------------------- -module(tls_dtls_connection). @@ -113,13 +115,11 @@ handle_session(#server_hello{cipher_suite = CipherSuite, PremasterSecret = make_premaster_secret(ReqVersion, KeyAlgorithm), - {ExpectNPN, Protocol} = case Protocol0 of - undefined -> - - {false, CurrentProtocol}; - _ -> - {ProtoExt =:= npn, Protocol0} - end, + {ExpectNPN, Protocol} = + case Protocol0 of + undefined -> {false, CurrentProtocol}; + _ -> {ProtoExt =:= npn, Protocol0} + end, State = State0#state{connection_states = ConnectionStates, handshake_env = HsEnv#handshake_env{kex_algorithm = KeyAlgorithm, @@ -163,10 +163,9 @@ hello(Type, Event, State) -> #hello_request{} | term(), #state{}) -> gen_statem:state_function_result(). %%-------------------------------------------------------------------- -user_hello({call, From}, cancel, #state{connection_env = #connection_env{negotiated_version = Version}} = State) -> +user_hello({call, From}, cancel, _State) -> gen_statem:reply(From, ok), - ssl_gen_statem:handle_own_alert(?ALERT_REC(?FATAL, ?USER_CANCELED, user_canceled), - Version, ?FUNCTION_NAME, State); + throw(?ALERT_REC(?FATAL, ?USER_CANCELED, user_canceled)); user_hello({call, From}, {handshake_continue, NewOptions, Timeout}, #state{static_env = #static_env{role = Role}, handshake_env = #handshake_env{hello = Hello}, @@ -209,7 +208,7 @@ abbreviated(internal, #finished{verify_data = Data} = Finished, Connection), Connection:next_event(connection, Record, State, [{{timeout, handshake}, infinity, close}]); #alert{} = Alert -> - ssl_gen_statem:handle_own_alert(Alert, Version, ?FUNCTION_NAME, State0) + throw(Alert) end; abbreviated(internal, #finished{verify_data = Data} = Finished, #state{static_env = #static_env{role = client, @@ -232,7 +231,7 @@ abbreviated(internal, #finished{verify_data = Data} = Finished, Connection), Connection:next_event(connection, Record, State, [{{timeout, handshake}, infinity, close} | Actions]); #alert{} = Alert -> - ssl_gen_statem:handle_own_alert(Alert, Version, ?FUNCTION_NAME, State0) + throw(Alert) end; %% only allowed to send next_protocol message after change cipher spec %% & before finished message and it is not allowed during renegotiation @@ -309,12 +308,8 @@ certify(info, Msg, State) -> handle_info(Msg, ?FUNCTION_NAME, State); certify(internal, #certificate{asn1_certificates = []}, #state{static_env = #static_env{role = server}, - connection_env = #connection_env{negotiated_version = Version}, - ssl_options = #{verify := verify_peer, - fail_if_no_peer_cert := true}} = - State) -> - Alert = ?ALERT_REC(?FATAL,?HANDSHAKE_FAILURE, no_client_certificate_provided), - ssl_gen_statem:handle_own_alert(Alert, Version, ?FUNCTION_NAME, State); + ssl_options = #{verify := verify_peer, fail_if_no_peer_cert := true}}) -> + throw(?ALERT_REC(?FATAL,?HANDSHAKE_FAILURE, no_client_certificate_provided)); certify(internal, #certificate{asn1_certificates = []}, #state{static_env = #static_env{role = server, protocol_cb = Connection}, @@ -324,11 +319,8 @@ certify(internal, #certificate{asn1_certificates = []}, Connection:next_event(?FUNCTION_NAME, no_record, State0#state{client_certificate_requested = false}); certify(internal, #certificate{}, #state{static_env = #static_env{role = server}, - connection_env = #connection_env{negotiated_version = Version}, - ssl_options = #{verify := verify_none}} = - State) -> - Alert = ?ALERT_REC(?FATAL,?UNEXPECTED_MESSAGE, unrequested_certificate), - ssl_gen_statem:handle_own_alert(Alert, Version, ?FUNCTION_NAME, State); + ssl_options = #{verify := verify_none}}) -> + throw(?ALERT_REC(?FATAL,?UNEXPECTED_MESSAGE, unrequested_certificate)); certify(internal, #certificate{}, #state{static_env = #static_env{protocol_cb = Connection}, handshake_env = #handshake_env{ @@ -355,7 +347,7 @@ certify(internal, #certificate{asn1_certificates = [Peer|_]} = Cert, handle_peer_cert(Role, PeerCert, PublicKeyInfo, State#state{client_certificate_requested = false}, Connection, []); #alert{} = Alert -> - ssl_gen_statem:handle_own_alert(Alert, Version, ?FUNCTION_NAME, State) + throw(Alert) end; certify(internal, #server_key_exchange{exchange_keys = Keys}, #state{static_env = #static_env{role = client, @@ -397,14 +389,12 @@ certify(internal, #server_key_exchange{exchange_keys = Keys}, session = session_handle_params(Params#server_key_params.params, Session)}, Connection); false -> - ssl_gen_statem:handle_own_alert(?ALERT_REC(?FATAL, ?DECRYPT_ERROR), - Version, ?FUNCTION_NAME, State) + throw(?ALERT_REC(?FATAL, ?DECRYPT_ERROR)) end end; certify(internal, #certificate_request{}, #state{static_env = #static_env{role = client}, - handshake_env = #handshake_env{kex_algorithm = KexAlg}, - connection_env = #connection_env{negotiated_version = Version}} = State) + handshake_env = #handshake_env{kex_algorithm = KexAlg}}) when KexAlg == dh_anon; KexAlg == ecdh_anon; KexAlg == psk; @@ -414,8 +404,7 @@ certify(internal, #certificate_request{}, KexAlg == srp_dss; KexAlg == srp_rsa; KexAlg == srp_anon -> - ssl_gen_statem:handle_own_alert(?ALERT_REC(?FATAL, ?HANDSHAKE_FAILURE), - Version, ?FUNCTION_NAME, State); + throw(?ALERT_REC(?FATAL, ?HANDSHAKE_FAILURE)); certify(internal, #certificate_request{}, #state{static_env = #static_env{role = client, protocol_cb = Connection}, @@ -443,7 +432,7 @@ certify(internal, #certificate_request{} = CertRequest, case ssl_handshake:select_hashsign(CertRequest, Cert, SupportedHashSigns, TLSVersion) of #alert {} = Alert -> - ssl_gen_statem:handle_own_alert(Alert, Version, ?FUNCTION_NAME, State); + throw(Alert); SelectedHashSign -> Connection:next_event(?FUNCTION_NAME, no_record, State#state{client_certificate_requested = true, @@ -455,7 +444,6 @@ certify(internal, #server_hello_done{}, #state{static_env = #static_env{role = client, protocol_cb = Connection}, session = #session{master_secret = undefined}, - connection_env = #connection_env{negotiated_version = Version}, handshake_env = #handshake_env{kex_algorithm = KexAlg, premaster_secret = undefined, server_psk_identity = PSKIdentity} = HsEnv, @@ -463,7 +451,7 @@ certify(internal, #server_hello_done{}, when KexAlg == psk -> case ssl_handshake:premaster_secret({KexAlg, PSKIdentity}, PSKLookup) of #alert{} = Alert -> - ssl_gen_statem:handle_own_alert(Alert, Version, ?FUNCTION_NAME, State0); + throw(Alert); PremasterSecret -> State = master_secret(PremasterSecret, State0#state{handshake_env = @@ -473,7 +461,7 @@ certify(internal, #server_hello_done{}, certify(internal, #server_hello_done{}, #state{static_env = #static_env{role = client, protocol_cb = Connection}, - connection_env = #connection_env{negotiated_version = {Major, Minor}} = Version, + connection_env = #connection_env{negotiated_version = {Major, Minor}}, handshake_env = #handshake_env{kex_algorithm = KexAlg, premaster_secret = undefined, server_psk_identity = PSKIdentity} = HsEnv, @@ -485,7 +473,7 @@ certify(internal, #server_hello_done{}, case ssl_handshake:premaster_secret({KexAlg, PSKIdentity}, PSKLookup, RSAPremasterSecret) of #alert{} = Alert -> - ssl_gen_statem:handle_own_alert(Alert, Version, ?FUNCTION_NAME, State0); + throw(Alert); PremasterSecret -> State = master_secret(PremasterSecret, State0#state{handshake_env = @@ -506,7 +494,7 @@ certify(internal, #server_hello_done{}, State = State0#state{connection_states = ConnectionStates}, client_certify_and_key_exchange(State, Connection); #alert{} = Alert -> - ssl_gen_statem:handle_own_alert(Alert, Version, ?FUNCTION_NAME, State0) + throw(Alert) end; %% Master secret is calculated from premaster_secret certify(internal, #server_hello_done{}, @@ -524,16 +512,14 @@ certify(internal, #server_hello_done{}, session = Session}, client_certify_and_key_exchange(State, Connection); #alert{} = Alert -> - ssl_gen_statem:handle_own_alert(Alert, Version, ?FUNCTION_NAME, State0) + throw(Alert) end; certify(internal = Type, #client_key_exchange{} = Msg, #state{static_env = #static_env{role = server}, client_certificate_requested = true, - connection_env = #connection_env{negotiated_version = Version}, - ssl_options = #{fail_if_no_peer_cert := true}} = State) -> + ssl_options = #{fail_if_no_peer_cert := true}}) -> %% We expect a certificate here - Alert = ?ALERT_REC(?FATAL,?UNEXPECTED_MESSAGE, {unexpected_msg, {Type, Msg}}), - ssl_gen_statem:handle_own_alert(Alert, Version, ?FUNCTION_NAME, State); + throw(?ALERT_REC(?FATAL,?UNEXPECTED_MESSAGE, {unexpected_msg, {Type, Msg}})); certify(internal, #client_key_exchange{exchange_keys = Keys}, State = #state{handshake_env = #handshake_env{kex_algorithm = KeyAlg}, static_env = #static_env{protocol_cb = Connection}, @@ -543,7 +529,7 @@ certify(internal, #client_key_exchange{exchange_keys = Keys}, State, Connection) catch #alert{} = Alert -> - ssl_gen_statem:handle_own_alert(Alert, Version, ?FUNCTION_NAME, State) + throw(Alert) end; certify(internal, #hello_request{}, _) -> keep_state_and_data; @@ -580,15 +566,14 @@ cipher(internal, #certificate_verify{signature = Signature, Connection:next_event(?FUNCTION_NAME, no_record, State#state{handshake_env = HsEnv#handshake_env{cert_hashsign_algorithm = HashSign}}); #alert{} = Alert -> - ssl_gen_statem:handle_own_alert(Alert, Version, ?FUNCTION_NAME, State) + throw(Alert) end; %% client must send a next protocol message if we are expecting it cipher(internal, #finished{}, #state{static_env = #static_env{role = server}, handshake_env = #handshake_env{expecting_next_protocol_negotiation = true, - negotiated_protocol = undefined}, - connection_env = #connection_env{negotiated_version = Version}} = State0) -> - ssl_gen_statem:handle_own_alert(?ALERT_REC(?FATAL,?UNEXPECTED_MESSAGE), Version, ?FUNCTION_NAME, State0); + negotiated_protocol = undefined}}) -> + throw(?ALERT_REC(?FATAL,?UNEXPECTED_MESSAGE)); cipher(internal, #finished{verify_data = Data} = Finished, #state{static_env = #static_env{role = Role, host = Host, @@ -610,7 +595,7 @@ cipher(internal, #finished{verify_data = Data} = Finished, cipher_role(Role, Data, Session, State#state{handshake_env = HsEnv#handshake_env{expecting_finished = false}}); #alert{} = Alert -> - ssl_gen_statem:handle_own_alert(Alert, Version, ?FUNCTION_NAME, State) + throw(Alert) end; %% only allowed to send next_protocol message after change cipher spec %% & before finished message and it is not allowed during renegotiation @@ -688,16 +673,10 @@ connection(Type, Event, State) -> downgrade(Type, Event, State) -> ssl_gen_statem:handle_common_event(Type, Event, ?FUNCTION_NAME, State). -gen_handshake(StateName, Type, Event, - #state{connection_env = #connection_env{negotiated_version = Version}} = State) -> - try tls_dtls_connection:StateName(Type, Event, State) of - Result -> - Result - catch - _:_ -> - ssl_gen_statem:handle_own_alert(?ALERT_REC(?FATAL, ?HANDSHAKE_FAILURE, - malformed_handshake_data), - Version, StateName, State) +gen_handshake(StateName, Type, Event, State) -> + try tls_dtls_connection:StateName(Type, Event, State) + catch error:_ -> + throw(?ALERT_REC(?FATAL, ?HANDSHAKE_FAILURE, malformed_handshake_data)) end. %%-------------------------------------------------------------------- @@ -825,20 +804,13 @@ new_server_hello(#server_hello{cipher_suite = CipherSuite, compression_method = Compression, session_id = SessionId}, #state{session = Session0, - static_env = #static_env{protocol_cb = Connection}, - connection_env = #connection_env{negotiated_version = Version}} = State0, Connection) -> - try server_certify_and_key_exchange(State0, Connection) of - #state{} = State1 -> - {State, Actions} = server_hello_done(State1, Connection), - Session = - Session0#session{session_id = SessionId, - cipher_suite = CipherSuite, - compression_method = Compression}, - Connection:next_event(certify, no_record, State#state{session = Session}, Actions) - catch - #alert{} = Alert -> - ssl_gen_statem:handle_own_alert(Alert, Version, hello, State0) - end. + static_env = #static_env{protocol_cb = Connection}} = State0, Connection) -> + #state{} = State1 = server_certify_and_key_exchange(State0, Connection), + {State, Actions} = server_hello_done(State1, Connection), + Session = Session0#session{session_id = SessionId, + cipher_suite = CipherSuite, + compression_method = Compression}, + Connection:next_event(certify, no_record, State#state{session = Session}, Actions). resumed_server_hello(#state{session = Session, connection_states = ConnectionStates0, @@ -854,7 +826,7 @@ resumed_server_hello(#state{session = Session, finalize_handshake(State1, abbreviated, Connection), Connection:next_event(abbreviated, no_record, State, Actions); #alert{} = Alert -> - ssl_gen_statem:handle_own_alert(Alert, Version, hello, State0) + throw(Alert) end. server_hello(ServerHello, State0, Connection) -> @@ -923,19 +895,11 @@ verify_client_cert(#state{static_env = #static_env{role = client}, verify_client_cert(#state{client_certificate_requested = false} = State, _) -> State. -client_certify_and_key_exchange(#state{connection_env = #connection_env{negotiated_version = Version}} = - State0, Connection) -> - try do_client_certify_and_key_exchange(State0, Connection) of - State1 = #state{} -> - {State2, Actions} = finalize_handshake(State1, certify, Connection), - State = State2#state{ - %% Reinitialize - client_certificate_requested = false}, - Connection:next_event(cipher, no_record, State, Actions) - catch - throw:#alert{} = Alert -> - ssl_gen_statem:handle_own_alert(Alert, Version, certify, State0) - end. +client_certify_and_key_exchange(State0, Connection) -> + State1 = do_client_certify_and_key_exchange(State0, Connection), + {State2, Actions} = finalize_handshake(State1, certify, Connection), + State = State2#state{client_certificate_requested = false}, %% Reinitialize + Connection:next_event(cipher, no_record, State, Actions). do_client_certify_and_key_exchange(State0, Connection) -> State1 = certify_client(State0, Connection), @@ -965,10 +929,10 @@ certify_client_key_exchange(#encrypted_premaster_secret{premaster_secret= EncPMS end; _ -> %% erlang:byte_size(Secret) =/= ?NUM_OF_PREMASTERSECRET_BYTES FakeSecret - catch + catch #alert{description = ?DECRYPT_ERROR} -> FakeSecret - end, + end, calculate_master_secret(PremasterSecret, State, Connection, certify, cipher); certify_client_key_exchange(#client_diffie_hellman_public{dh_public = ClientPublicDhKey}, #state{handshake_env = #handshake_env{diffie_hellman_params = #'DHParameter'{} = Params, @@ -1189,12 +1153,7 @@ key_exchange(#state{static_env = #static_env{role = server}, KexAlg == srp_rsa; KexAlg == srp_anon -> SrpParams = handle_srp_identity(Username, LookupFun), - Keys = case generate_srp_server_keys(SrpParams, 0) of - Alert = #alert{} -> - throw(Alert); - Keys0 = {_,_} -> - Keys0 - end, + Keys = generate_srp_server_keys(SrpParams, 0), #{security_parameters := SecParams} = ssl_record:pending_connection_state(ConnectionStates0, read), #security_parameters{client_random = ClientRandom, @@ -1301,7 +1260,7 @@ rsa_key_exchange(Version, PremasterSecret, PublicKeyInfo = {Algorithm, _, _}) {premaster_secret, PremasterSecret, PublicKeyInfo}); rsa_key_exchange(_, _, _) -> - throw (?ALERT_REC(?FATAL,?HANDSHAKE_FAILURE, pub_key_is_not_rsa)). + throw(?ALERT_REC(?FATAL,?HANDSHAKE_FAILURE, pub_key_is_not_rsa)). rsa_psk_key_exchange(Version, PskIdentity, PremasterSecret, PublicKeyInfo = {Algorithm, _, _}) @@ -1318,7 +1277,7 @@ rsa_psk_key_exchange(Version, PskIdentity, PremasterSecret, {psk_premaster_secret, PskIdentity, PremasterSecret, PublicKeyInfo}); rsa_psk_key_exchange(_, _, _, _) -> - throw (?ALERT_REC(?FATAL,?HANDSHAKE_FAILURE, pub_key_is_not_rsa)). + throw(?ALERT_REC(?FATAL,?HANDSHAKE_FAILURE, pub_key_is_not_rsa)). request_client_cert(#state{handshake_env = #handshake_env{kex_algorithm = Alg}} = State, _) when Alg == dh_anon; @@ -1366,7 +1325,7 @@ calculate_master_secret(PremasterSecret, session = Session}, Connection:next_event(Next, no_record, State); #alert{} = Alert -> - ssl_gen_statem:handle_own_alert(Alert, Version, certify, State0) + throw(Alert) end. finalize_handshake(State0, StateName, Connection) -> @@ -1480,7 +1439,7 @@ calculate_secret(#server_srp_params{srp_n = Prime, srp_g = Generator} = ServerKe certify, certify). master_secret(#alert{} = Alert, _) -> - Alert; + throw(Alert); master_secret(PremasterSecret, #state{static_env = #static_env{role = Role}, connection_env = #connection_env{negotiated_version = Version}, session = Session, @@ -1493,11 +1452,11 @@ master_secret(PremasterSecret, #state{static_env = #static_env{role = Role}, Session#session{master_secret = MasterSecret}, connection_states = ConnectionStates}; #alert{} = Alert -> - Alert + throw(Alert) end. generate_srp_server_keys(_SrpParams, 10) -> - ?ALERT_REC(?FATAL, ?ILLEGAL_PARAMETER); + throw(?ALERT_REC(?FATAL, ?ILLEGAL_PARAMETER)); generate_srp_server_keys(SrpParams = #srp_user{generator = Generator, prime = Prime, verifier = Verifier}, N) -> @@ -1510,9 +1469,8 @@ generate_srp_server_keys(SrpParams = end. generate_srp_client_keys(_Generator, _Prime, 10) -> - ?ALERT_REC(?FATAL, ?ILLEGAL_PARAMETER); + throw(?ALERT_REC(?FATAL, ?ILLEGAL_PARAMETER)); generate_srp_client_keys(Generator, Prime, N) -> - try crypto:generate_key(srp, {user, [Generator, Prime, '6a']}) of Keys -> Keys @@ -1645,7 +1603,7 @@ handle_resumed_session(SessId, #state{static_env = #static_env{host = Host, connection_states = ConnectionStates, session = Session}); #alert{} = Alert -> - ssl_gen_statem:handle_own_alert(Alert, Version, hello, State) + throw(Alert) end. make_premaster_secret({MajVer, MinVer}, rsa) -> @@ -1674,8 +1632,8 @@ handle_sni_extension(#state{static_env = case ssl_gen_statem:handle_sni_extension(PossibleSNI, State0) of {ok, State} -> State; - {error, Alert} -> - Alert + {error, #alert{}=Alert} -> + throw(Alert) end. ensure_tls({254, _} = Version) -> diff --git a/lib/ssl/src/tls_dyn_connection_sup.erl b/lib/ssl/src/tls_dyn_connection_sup.erl new file mode 100644 index 0000000000..cb0576cbd6 --- /dev/null +++ b/lib/ssl/src/tls_dyn_connection_sup.erl @@ -0,0 +1,78 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 2021-2021. All Rights Reserved. +%% +%% Licensed under the Apache License, Version 2.0 (the "License"); +%% you may not use this file except in compliance with the License. +%% You may obtain a copy of the License at +%% +%% http://www.apache.org/licenses/LICENSE-2.0 +%% +%% Unless required by applicable law or agreed to in writing, software +%% distributed under the License is distributed on an "AS IS" BASIS, +%% WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +%% See the License for the specific language governing permissions and +%% limitations under the License. +%% +%% %CopyrightEnd% +%% + +%% +%%---------------------------------------------------------------------- +%% Purpose: Supervises the TLS generic state machine, a process that +%% owns the transport socket and hence is a significant child, and the +%% corresponding TLS sender process that sends data to avoid blocking +%% the state machine process. +%% ---------------------------------------------------------------------- + +-module(tls_dyn_connection_sup). + +-behaviour(supervisor). + +%% API +-export([start_link/0]). +-export([start_child/3]). + +%% Supervisor callback +-export([init/1]). + +%%%========================================================================= +%%% API +%%%========================================================================= +start_link() -> + supervisor:start_link(?MODULE, []). + +start_child(Sup, sender, Args) -> + supervisor:start_child(Sup, sender(Args)); +start_child(Sup, receiver, Args) -> + supervisor:start_child(Sup, receiver(Args)). + +%%%========================================================================= +%%% Supervisor callback +%%%========================================================================= +init(_) -> + SupFlags = #{strategy => one_for_all, + auto_shutdown => any_significant, + intensity => 0, + period => 3600 + }, + ChildSpecs = [], + {ok, {SupFlags, ChildSpecs}}. + +sender(Args) -> + #{id => sender, + restart => temporary, + type => worker, + start => {tls_sender, start_link, Args}, + modules => [tls_sender] + }. + +receiver(Args) -> + #{id => receiver, + restart => temporary, + type => worker, + significant => true, + start => {ssl_gen_statem, start_link, Args}, + modules => [ssl_gen_statem, tls_connection, tls_connection_1_3] + }. diff --git a/lib/ssl/src/tls_gen_connection.erl b/lib/ssl/src/tls_gen_connection.erl index 217620f62e..59d30d7cb0 100644 --- a/lib/ssl/src/tls_gen_connection.erl +++ b/lib/ssl/src/tls_gen_connection.erl @@ -82,9 +82,10 @@ start_fsm(Role, Host, Port, Socket, {#{erl_dist := false},_, Trackers} = Opts, User, {CbModule, _,_, _, _} = CbInfo, Timeout) -> try - {ok, Sender} = tls_sender:start(), - {ok, Pid} = tls_connection_sup:start_child([Role, Sender, Host, Port, Socket, - Opts, User, CbInfo]), + {ok, DynSup} = tls_connection_sup:start_child([]), + {ok, Sender} = tls_dyn_connection_sup:start_child(DynSup, sender, []), + {ok, Pid} = tls_dyn_connection_sup:start_child(DynSup, receiver, [Role, Sender, Host, Port, Socket, + Opts, User, CbInfo]), {ok, SslSocket} = ssl_gen_statem:socket_control(?MODULE, Socket, [Pid, Sender], CbModule, Trackers), ssl_gen_statem:handshake(SslSocket, Timeout) catch @@ -96,9 +97,10 @@ start_fsm(Role, Host, Port, Socket, {#{erl_dist := true},_, Trackers} = Opts, User, {CbModule, _,_, _, _} = CbInfo, Timeout) -> try - {ok, Sender} = tls_sender:start([{spawn_opt, ?DIST_CNTRL_SPAWN_OPTS}]), - {ok, Pid} = tls_connection_sup:start_child_dist([Role, Sender, Host, Port, Socket, - Opts, User, CbInfo]), + {ok, DynSup} = tls_connection_sup:start_child_dist([]), + {ok, Sender} = tls_dyn_connection_sup:start_child(DynSup, sender, [[{spawn_opt, ?DIST_CNTRL_SPAWN_OPTS}]]), + {ok, Pid} = tls_dyn_connection_sup:start_child(DynSup, receiver, [Role, Sender, Host, Port, Socket, + Opts, User, CbInfo]), {ok, SslSocket} = ssl_gen_statem:socket_control(?MODULE, Socket, [Pid, Sender], CbModule, Trackers), ssl_gen_statem:handshake(SslSocket, Timeout) catch @@ -241,13 +243,12 @@ getopts(Transport, Socket, Tag) -> %% raw data from socket, upack records handle_info({Protocol, _, Data}, StateName, - #state{static_env = #static_env{data_tag = Protocol}, - connection_env = #connection_env{negotiated_version = Version}} = State0) -> + #state{static_env = #static_env{data_tag = Protocol}} = State0) -> case next_tls_record(Data, StateName, State0) of {Record, State} -> next_event(StateName, Record, State); #alert{} = Alert -> - ssl_gen_statem:handle_own_alert(Alert, Version, StateName, State0) + ssl_gen_statem:handle_own_alert(Alert, StateName, State0) end; handle_info({PassiveTag, Socket}, StateName, #state{static_env = #static_env{socket = Socket, @@ -321,9 +322,6 @@ handle_info({CloseTag, Socket}, StateName, %% is called after all data has been deliver. {next_state, StateName, State#state{protocol_specific = PS#{active_n_toggle => true}}, []} end; -handle_info({'EXIT', Sender, Reason}, _, - #state{protocol_specific = #{sender := Sender}} = State) -> - {stop, {shutdown, {sender_died, Reason}}, State}; handle_info(Msg, StateName, State) -> ssl_gen_statem:handle_info(Msg, StateName, State). @@ -404,26 +402,25 @@ handle_protocol_record(#ssl_tls{type = ?HANDSHAKE, fragment = Data}, end end catch throw:#alert{} = Alert -> - ssl_gen_statem:handle_own_alert(Alert, Version, StateName, State0) + ssl_gen_statem:handle_own_alert(Alert, StateName, State0) end; %%% TLS record protocol level change cipher messages handle_protocol_record(#ssl_tls{type = ?CHANGE_CIPHER_SPEC, fragment = Data}, StateName, State) -> {next_state, StateName, State, [{next_event, internal, #change_cipher_spec{type = Data}}]}; %%% TLS record protocol level Alert messages -handle_protocol_record(#ssl_tls{type = ?ALERT, fragment = EncAlerts}, StateName, - #state{connection_env = #connection_env{negotiated_version = Version}} = State) -> +handle_protocol_record(#ssl_tls{type = ?ALERT, fragment = EncAlerts}, StateName,State) -> try decode_alerts(EncAlerts) of Alerts = [_|_] -> handle_alerts(Alerts, {next_state, StateName, State}); [] -> ssl_gen_statem:handle_own_alert(?ALERT_REC(?FATAL, ?HANDSHAKE_FAILURE, empty_alert), - Version, StateName, State); + StateName, State); #alert{} = Alert -> - ssl_gen_statem:handle_own_alert(Alert, Version, StateName, State) + ssl_gen_statem:handle_own_alert(Alert, StateName, State) catch _:_ -> ssl_gen_statem:handle_own_alert(?ALERT_REC(?FATAL, ?HANDSHAKE_FAILURE, alert_decode_error), - Version, StateName, State) + StateName, State) end; %% Ignore unknown TLS record level protocol messages diff --git a/lib/ssl/src/tls_handshake.erl b/lib/ssl/src/tls_handshake.erl index 05cf1a7339..cb00fa57a3 100644 --- a/lib/ssl/src/tls_handshake.erl +++ b/lib/ssl/src/tls_handshake.erl @@ -108,7 +108,8 @@ client_hello(_Host, _Port, ConnectionStates, boolean(), #session{}) -> {tls_record:tls_version(), ssl:session_id(), ssl_record:connection_states(), alpn | npn, binary() | undefined, map()}| - {atom(), atom(), tls_record:tls_version(), map()} | #alert{}. + {atom(), atom(), tls_record:tls_version(), map()}. + % Otherwise Throws #alert{} %% %% Description: Handles a received hello message %%-------------------------------------------------------------------- @@ -128,7 +129,7 @@ hello(#server_hello{server_version = {Major, Minor}, (M > 3 orelse M =:= 3 andalso N >= 4) andalso %% TLS 1.3 client (Major =:= 3 andalso Minor < 3 andalso %% Negotiating TLS 1.1 or prior Down =:= ?RANDOM_OVERRIDE_TLS11) -> - ?ALERT_REC(?FATAL, ?ILLEGAL_PARAMETER); + throw(?ALERT_REC(?FATAL, ?ILLEGAL_PARAMETER)); %% TLS 1.2 clients SHOULD also check that the last eight bytes are not %% equal to the second value if the ServerHello indicates TLS 1.1 or below. @@ -138,7 +139,7 @@ hello(#server_hello{server_version = {Major, Minor}, when (M =:= 3 andalso N =:= 3) andalso %% TLS 1.2 client (Major =:= 3 andalso Minor < 3 andalso %% Negotiating TLS 1.1 or prior Down =:= ?RANDOM_OVERRIDE_TLS11) -> - ?ALERT_REC(?FATAL, ?ILLEGAL_PARAMETER); + throw(?ALERT_REC(?FATAL, ?ILLEGAL_PARAMETER)); %% TLS 1.3 - 4.2.1. Supported Versions @@ -157,9 +158,8 @@ hello(#server_hello{server_version = LegacyVersion, compression_method = Compression, session_id = SessionId, extensions = #{server_hello_selected_version := - #server_hello_selected_version{selected_version = Version} = HelloExt} - }, - #{versions := SupportedVersions, + #server_hello_selected_version{selected_version = Version}} = HelloExt}, + #{versions := SupportedVersions, ocsp_stapling := Stapling} = SslOpt, ConnectionStates0, Renegotiation, OldId) -> %% In TLS 1.3, the TLS server indicates its version using the "supported_versions" extension @@ -169,7 +169,7 @@ hello(#server_hello{server_version = LegacyVersion, case LegacyVersion > {3,3} orelse LegacyVersion =:= {3,3} andalso Version < {3,3} of true -> - ?ALERT_REC(?FATAL, ?ILLEGAL_PARAMETER); + throw(?ALERT_REC(?FATAL, ?ILLEGAL_PARAMETER)); false -> case tls_record:is_acceptable_version(Version, SupportedVersions) of true -> @@ -186,7 +186,7 @@ hello(#server_hello{server_version = LegacyVersion, ocsp_expect => ocsp_expect(Stapling)}} end; false -> - ?ALERT_REC(?FATAL, ?ILLEGAL_PARAMETER) + throw(?ALERT_REC(?FATAL, ?ILLEGAL_PARAMETER)) end end; @@ -205,7 +205,7 @@ hello(#server_hello{server_version = Version, Compression, HelloExt, SslOpt, ConnectionStates0, Renegotiation, IsNew); false -> - ?ALERT_REC(?FATAL, ?PROTOCOL_VERSION) + throw(?ALERT_REC(?FATAL, ?PROTOCOL_VERSION)) end. @@ -219,7 +219,7 @@ hello(#server_hello{server_version = Version, {tls_record:tls_version(), {resumed | new, #session{}}, ssl_record:connection_states(), binary() | undefined, HelloExt::map(), {ssl:hash(), ssl:sign_algo()} | - undefined} | {atom(), atom()} | {atom(), atom(), tuple()} | #alert{}. + undefined} | {atom(), atom()} | {atom(), atom(), tuple()}. %% TLS 1.2 Server %% - If "supported_versions" is present (ClientHello): %% - Select version from "supported_versions" (ignore ClientHello.legacy_version) @@ -245,8 +245,8 @@ hello(#client_hello{client_version = _ClientVersion, Version = ssl_handshake:select_supported_version(ClientVersions, Versions), do_hello(Version, Versions, CipherSuites, Hello, SslOpts, Info, Renegotiation) catch - _:_ -> - ?ALERT_REC(?FATAL, ?HANDSHAKE_FAILURE, malformed_handshake_data) + error:_ -> + throw(?ALERT_REC(?FATAL, ?HANDSHAKE_FAILURE, malformed_handshake_data)) end; hello(#client_hello{client_version = ClientVersion, @@ -259,9 +259,9 @@ hello(#client_hello{client_version = ClientVersion, catch error:{case_clause,{asn1, Asn1Reason}} -> %% ASN-1 decode of certificate somehow failed - ?ALERT_REC(?FATAL, ?INTERNAL_ERROR, {failed_to_decode_own_certificate, Asn1Reason}); - _:_ -> - ?ALERT_REC(?FATAL, ?HANDSHAKE_FAILURE, malformed_handshake_data) + throw(?ALERT_REC(?FATAL, ?INTERNAL_ERROR, {failed_to_decode_own_certificate, Asn1Reason})); + error:_ -> + throw(?ALERT_REC(?FATAL, ?HANDSHAKE_FAILURE, malformed_handshake_data)) end. %%-------------------------------------------------------------------- @@ -337,13 +337,13 @@ handle_client_hello(Version, ClientHashSigns, SupportedHashSigns, OwnCert, Version), ECCCurve = ssl_handshake:select_curve(Curves, SupportedECCs, ECCOrder), {Type, #session{cipher_suite = CipherSuite} = Session1} - = ssl_handshake:select_session(SugesstedId, CipherSuites, + = ssl_handshake:select_session(SugesstedId, CipherSuites, AvailableHashSigns, Compressions, SessIdTracker, Session0#session{ecc = ECCCurve}, Version, SslOpts, OwnCert), - case CipherSuite of + case CipherSuite of no_suite -> - ?ALERT_REC(?FATAL, ?INSUFFICIENT_SECURITY, no_suitable_ciphers); + throw(?ALERT_REC(?FATAL, ?INSUFFICIENT_SECURITY, no_suitable_ciphers)); _ -> #{key_exchange := KeyExAlg} = ssl_cipher_format:suite_bin_to_map(CipherSuite), case ssl_handshake:select_hashsign({ClientHashSigns, ClientSignatureSchemes}, @@ -351,7 +351,7 @@ handle_client_hello(Version, SupportedHashSigns, Version) of #alert{} = Alert -> - Alert; + throw(Alert); HashSign -> handle_client_hello_extensions(Version, Type, Random, CipherSuites, HelloExt, @@ -361,47 +361,38 @@ handle_client_hello(Version, end end; false -> - ?ALERT_REC(?FATAL, ?PROTOCOL_VERSION) + throw(?ALERT_REC(?FATAL, ?PROTOCOL_VERSION)) end. handle_client_hello_extensions(Version, Type, Random, CipherSuites, HelloExt, SslOpts, Session0, ConnectionStates0, Renegotiation, HashSign) -> - try ssl_handshake:handle_client_hello_extensions(tls_record, Random, CipherSuites, - HelloExt, Version, SslOpts, - Session0, ConnectionStates0, + {Session, ConnectionStates, Protocol, ServerHelloExt} = + ssl_handshake:handle_client_hello_extensions(tls_record, Random, CipherSuites, + HelloExt, Version, SslOpts, + Session0, ConnectionStates0, Renegotiation, - Session0#session.is_resumable) of - {Session, ConnectionStates, Protocol, ServerHelloExt} -> - {Version, {Type, Session}, ConnectionStates, Protocol, - ServerHelloExt, HashSign} - catch throw:Alert -> - Alert - end. - + Session0#session.is_resumable), + {Version, {Type, Session}, ConnectionStates, Protocol, ServerHelloExt, HashSign}. handle_server_hello_extensions(Version, SessionId, Random, CipherSuite, - Compression, HelloExt, SslOpt, ConnectionStates0, Renegotiation, IsNew) -> - try ssl_handshake:handle_server_hello_extensions(tls_record, Random, CipherSuite, - Compression, HelloExt, Version, - SslOpt, ConnectionStates0, - Renegotiation, IsNew) of - {ConnectionStates, ProtoExt, Protocol, OcspState} -> - {Version, SessionId, ConnectionStates, ProtoExt, Protocol, OcspState} - catch throw:Alert -> - Alert - end. - + Compression, HelloExt, SslOpt, ConnectionStates0, Renegotiation, IsNew) -> + {ConnectionStates, ProtoExt, Protocol, OcspState} = + ssl_handshake:handle_server_hello_extensions(tls_record, Random, CipherSuite, + Compression, HelloExt, Version, + SslOpt, ConnectionStates0, + Renegotiation, IsNew), + {Version, SessionId, ConnectionStates, ProtoExt, Protocol, OcspState}. do_hello(undefined, _Versions, _CipherSuites, _Hello, _SslOpts, _Info, _Renegotiation) -> - ?ALERT_REC(?FATAL, ?PROTOCOL_VERSION); + throw(?ALERT_REC(?FATAL, ?PROTOCOL_VERSION)); do_hello(Version, Versions, CipherSuites, Hello, SslOpts, Info, Renegotiation) -> case ssl_cipher:is_fallback(CipherSuites) of true -> Highest = tls_record:highest_protocol_version(Versions), case tls_record:is_higher(Highest, Version) of true -> - ?ALERT_REC(?FATAL, ?INAPPROPRIATE_FALLBACK); + throw(?ALERT_REC(?FATAL, ?INAPPROPRIATE_FALLBACK)); false -> handle_client_hello(Version, Hello, SslOpts, Info, Renegotiation) end; @@ -444,9 +435,7 @@ get_tls_handshake_aux(Version, <<?BYTE(Type), ?UINT24(Length), ssl_logger:debug(LogLevel, inbound, 'handshake', Handshake), get_tls_handshake_aux(Version, Rest, Opts, [{Handshake,Raw} | Acc]) catch - throw:#alert{} = Alert -> - throw(Alert); - _:_ -> + error:_ -> throw(?ALERT_REC(?FATAL, ?HANDSHAKE_FAILURE, handshake_decode_error)) end; get_tls_handshake_aux(_Version, Data, _, Acc) -> diff --git a/lib/ssl/src/tls_record.erl b/lib/ssl/src/tls_record.erl index a551718c38..30341f1598 100644 --- a/lib/ssl/src/tls_record.erl +++ b/lib/ssl/src/tls_record.erl @@ -34,8 +34,8 @@ %% Handling of incoming data -export([get_tls_records/5, - init_connection_states/2, - init_connection_states/3]). + init_connection_states/3, + init_connection_states/4]). %% Encoding TLS records -export([encode_handshake/3, encode_alert_record/3, @@ -66,29 +66,35 @@ %% Handling of incoming data %%==================================================================== %%-------------------------------------------------------------------- --spec init_connection_states(Role, BeastMitigation) -> +-spec init_connection_states(Role, Version, BeastMitigation) -> ssl_record:connection_states() when Role :: client | server, + Version :: tls_version(), BeastMitigation :: one_n_minus_one | zero_n | disabled. -%% +%% %% Description: Creates a connection_states record with appropriate %% values for the initial SSL connection setup. %%-------------------------------------------------------------------- -init_connection_states(Role, BeastMitigation) -> +init_connection_states(Role, Version, BeastMitigation) -> MaxEarlyDataSize = ssl_config:get_max_early_data_size(), - init_connection_states(Role, BeastMitigation, MaxEarlyDataSize). + init_connection_states(Role, Version, BeastMitigation, MaxEarlyDataSize). %% --spec init_connection_states(Role, BeastMitigation, MaxEarlyDataSize) -> +-spec init_connection_states(Role, Version, BeastMitigation, + MaxEarlyDataSize) -> ssl_record:connection_states() when Role :: client | server, + Version :: tls_version(), BeastMitigation :: one_n_minus_one | zero_n | disabled, MaxEarlyDataSize :: non_neg_integer(). -init_connection_states(Role, BeastMitigation, MaxEarlyDataSize) -> +init_connection_states(Role, Version, BeastMitigation, MaxEarlyDataSize) -> ConnectionEnd = ssl_record:record_protocol_role(Role), Current = initial_connection_state(ConnectionEnd, BeastMitigation, MaxEarlyDataSize), - Pending = ssl_record:empty_connection_state(ConnectionEnd, BeastMitigation, MaxEarlyDataSize), + Pending = ssl_record:empty_connection_state(ConnectionEnd, + Version, + BeastMitigation, + MaxEarlyDataSize), #{current_read => Current, pending_read => Pending, current_write => Current, @@ -706,9 +712,7 @@ encode_fragments(Type, Version, [Text|Data], {MajVer, MinVer} = Version, CipherHeader = <<?BYTE(Type), ?BYTE(MajVer), ?BYTE(MinVer), ?UINT16(Length)>>, encode_fragments(Type, Version, Data, CS, CompS, CipherS, Seq + 1, - [[CipherHeader, CipherFragment] | CipherFragments]); -encode_fragments(_Type, _Version, _Data, CS, _CompS, _CipherS, _Seq, _CipherFragments) -> - exit({cs, CS}). + [[CipherHeader, CipherFragment] | CipherFragments]). %%-------------------------------------------------------------------- %% 1/n-1 splitting countermeasure Rizzo/Duong-Beast, RC4 ciphers are diff --git a/lib/ssl/src/tls_record_1_3.erl b/lib/ssl/src/tls_record_1_3.erl index 3e42e3bf97..92d6491e38 100644 --- a/lib/ssl/src/tls_record_1_3.erl +++ b/lib/ssl/src/tls_record_1_3.erl @@ -291,10 +291,7 @@ encode_plain_text(#inner_plaintext{ %% structures are written directly onto the wire. #tls_cipher_text{opaque_type = Type, legacy_version = {3,3}, - encoded_record = Data}; - -encode_plain_text(_, CS) -> - exit({cs, CS}). + encoded_record = Data}. additional_data(Length) -> <<?BYTE(?OPAQUE_TYPE), ?BYTE(3), ?BYTE(3),?UINT16(Length)>>. diff --git a/lib/ssl/src/tls_sender.erl b/lib/ssl/src/tls_sender.erl index 3d2cafa24c..065e5b9987 100644 --- a/lib/ssl/src/tls_sender.erl +++ b/lib/ssl/src/tls_sender.erl @@ -30,15 +30,30 @@ -include("tls_handshake_1_3.hrl"). %% API --export([start/0, start/1, initialize/2, send_data/2, - send_post_handshake/2, send_alert/2, - send_and_ack_alert/2, setopts/2, renegotiate/1, peer_renegotiate/1, downgrade/2, +-export([start_link/0, + start_link/1, + initialize/2, + send_data/2, + send_post_handshake/2, + send_alert/2, + send_and_ack_alert/2, + setopts/2, + renegotiate/1, + peer_renegotiate/1, + downgrade/2, update_connection_state/3, - dist_tls_socket/1, dist_handshake_complete/3]). + dist_tls_socket/1, + dist_handshake_complete/3]). %% gen_statem callbacks --export([callback_mode/0, init/1, terminate/3, code_change/4]). --export([init/3, connection/3, handshake/3, death_row/3]). +-export([callback_mode/0, + init/1, + terminate/3, + code_change/4]). +-export([init/3, + connection/3, + handshake/3, + death_row/3]). -record(static, {connection_pid, @@ -65,10 +80,10 @@ %%% API %%%=================================================================== %%-------------------------------------------------------------------- --spec start() -> {ok, Pid :: pid()} | +-spec start_link() -> {ok, Pid :: pid()} | ignore | {error, Error :: term()}. --spec start(list()) -> {ok, Pid :: pid()} | +-spec start_link(list()) -> {ok, Pid :: pid()} | ignore | {error, Error :: term()}. @@ -76,10 +91,10 @@ %% may happen when a socket is busy (busy port) and the %% same process is sending and receiving %%-------------------------------------------------------------------- -start() -> - gen_statem:start(?MODULE, [], []). -start(SpawnOpts) -> - gen_statem:start(?MODULE, [], SpawnOpts). +start_link() -> + gen_statem:start_link(?MODULE, [], []). +start_link(SpawnOpts) -> + gen_statem:start_link(?MODULE, [], SpawnOpts). %%-------------------------------------------------------------------- -spec initialize(pid(), map()) -> ok. @@ -613,7 +628,9 @@ call(FsmPid, Event) -> {error, closed}; exit:{normal, _} -> {error, closed}; - exit:{{shutdown, _},_} -> + exit:{shutdown,_} -> + {error, closed}; + exit:{{shutdown, _},_} -> {error, closed} end. diff --git a/lib/ssl/src/tls_server_session_ticket_sup.erl b/lib/ssl/src/tls_server_session_ticket_sup.erl index bdde94ecea..a515e8bbe0 100644 --- a/lib/ssl/src/tls_server_session_ticket_sup.erl +++ b/lib/ssl/src/tls_server_session_ticket_sup.erl @@ -59,18 +59,16 @@ sup_name(dist) -> %%%========================================================================= %%% Supervisor callback %%%========================================================================= -init(_O) -> - RestartStrategy = simple_one_for_one, - MaxR = 0, - MaxT = 3600, - - Name = undefined, % As simple_one_for_one is used. - StartFunc = {tls_server_session_ticket, start_link, []}, - Restart = temporary, % E.g. should not be restarted - Shutdown = 4000, - Modules = [tls_server_session_ticket], - Type = worker, - - ChildSpec = {Name, StartFunc, Restart, Shutdown, Type, Modules}, - {ok, {{RestartStrategy, MaxR, MaxT}, [ChildSpec]}}. - +init(_) -> + SupFlags = #{strategy => simple_one_for_one, + intensity => 0, + period => 3600 + }, + ChildSpecs = [#{id => undefined, + start => {tls_server_session_ticket, start_link, []}, + restart => transient, + shutdown => 4000, + modules => [tls_server_session_ticket], + type => worker + }], + {ok, {SupFlags, ChildSpecs}}. diff --git a/lib/ssl/src/tls_server_sup.erl b/lib/ssl/src/tls_server_sup.erl index b2f011f221..7f739ed015 100644 --- a/lib/ssl/src/tls_server_sup.erl +++ b/lib/ssl/src/tls_server_sup.erl @@ -43,18 +43,19 @@ start_link() -> %%% Supervisor callback %%%========================================================================= -init([]) -> - ListenTracker = listen_options_tracker_child_spec(), - SessionTracker = tls_server_session_child_spec(), - Pre_1_3SessionTracker = ssl_server_session_child_spec(), - Pre_1_3UpgradeSessionTracker = ssl_upgrade_server_session_child_spec(), - - {ok, {{one_for_all, 10, 3600}, [ListenTracker, - SessionTracker, - Pre_1_3SessionTracker, - Pre_1_3UpgradeSessionTracker - ]}}. - +init([]) -> + ChildSpecs = [listen_options_tracker_child_spec(), + tls_server_session_child_spec(), %% TLS-1.3 Session ticket handling + ssl_server_session_child_spec(), %% PRE TLS-1.3 session handling + ssl_upgrade_server_session_child_spec() %% PRE TLS-1.3 session handling for upgrade servers + ], + SupFlags = #{strategy => one_for_all, + intensity => 10, + period => 3600 + }, + {ok, {SupFlags, ChildSpecs}}. + + %%-------------------------------------------------------------------- %%% Internal functions @@ -63,37 +64,37 @@ init([]) -> %% Handles emulated options so that they inherited by the accept %% socket, even when setopts is performed on the listen socket listen_options_tracker_child_spec() -> - Name = tls_socket, - StartFunc = {ssl_listen_tracker_sup, start_link, []}, - Restart = permanent, - Shutdown = 4000, - Modules = [ssl_listen_tracker_sup], - Type = supervisor, - {Name, StartFunc, Restart, Shutdown, Type, Modules}. + #{id => ssl_listen_tracker_sup, + start => {ssl_listen_tracker_sup, start_link, []}, + restart => permanent, + shutdown => 4000, + modules => [ssl_listen_tracker_sup], + type => supervisor + }. tls_server_session_child_spec() -> - Name = tls_server_session_ticket, - StartFunc = {tls_server_session_ticket_sup, start_link, []}, - Restart = permanent, - Shutdown = 4000, - Modules = [tls_server_session_ticket_sup], - Type = supervisor, - {Name, StartFunc, Restart, Shutdown, Type, Modules}. + #{id => tls_server_session_ticket, + start => {tls_server_session_ticket_sup, start_link, []}, + restart => permanent, + shutdown => 4000, + modules => [tls_server_session_ticket_sup], + type => supervisor + }. ssl_server_session_child_spec() -> - Name = ssl_server_session_cache_sup, - StartFunc = {ssl_server_session_cache_sup, start_link, []}, - Restart = permanent, - Shutdown = 4000, - Modules = [ssl_server_session_cache_sup], - Type = supervisor, - {Name, StartFunc, Restart, Shutdown, Type, Modules}. + #{id => ssl_server_session_cache_sup, + start => {ssl_server_session_cache_sup, start_link, []}, + restart => permanent, + shutdown => 4000, + modules => [ssl_server_session_cache_sup], + type => supervisor + }. ssl_upgrade_server_session_child_spec() -> - Name = ssl_upgrade_server_session_cache_sup, - StartFunc = {ssl_upgrade_server_session_cache_sup, start_link, []}, - Restart = permanent, - Shutdown = 4000, - Modules = [ssl_upgrade_server_session_cache_sup], - Type = supervisor, - {Name, StartFunc, Restart, Shutdown, Type, Modules}. + #{id => ssl_upgrade_server_session_cache_sup, + start => {ssl_upgrade_server_session_cache_sup, start_link, []}, + restart => permanent, + shutdown => 4000, + modules => [ssl_upgrade_server_session_cache_sup], + type => supervisor + }. diff --git a/lib/ssl/src/tls_socket.erl b/lib/ssl/src/tls_socket.erl index 0a583dbb1d..12d59c6b4c 100644 --- a/lib/ssl/src/tls_socket.erl +++ b/lib/ssl/src/tls_socket.erl @@ -105,15 +105,7 @@ accept(ListenSocket, #config{transport_info = {Transport,_,_,_,_} = CbInfo, Tracker = proplists:get_value(option_tracker, Trackers), {ok, EmOpts} = get_emulated_opts(Tracker), {ok, Port} = tls_socket:port(Transport, Socket), - {ok, Sender} = tls_sender:start(), - ConnArgs = [server, Sender, "localhost", Port, Socket, - {SslOpts, emulated_socket_options(EmOpts, #socket_options{}), Trackers}, self(), CbInfo], - case tls_connection_sup:start_child(ConnArgs) of - {ok, Pid} -> - ssl_gen_statem:socket_control(ConnectionCb, Socket, [Pid, Sender], Transport, Trackers); - {error, Reason} -> - {error, Reason} - end; + start_tls_server_connection(SslOpts, ConnectionCb, Transport, Port, Socket, EmOpts, Trackers, CbInfo); {error, Reason} -> {error, Reason} end. @@ -399,6 +391,19 @@ code_change(_OldVsn, State, _Extra) -> call(Pid, Msg) -> gen_server:call(Pid, Msg, infinity). +start_tls_server_connection(SslOpts, ConnectionCb, Transport, Port, Socket, EmOpts, Trackers, CbInfo) -> + try + {ok, DynSup} = tls_connection_sup:start_child([]), + {ok, Sender} = tls_dyn_connection_sup:start_child(DynSup, sender, []), + ConnArgs = [server, Sender, "localhost", Port, Socket, + {SslOpts, emulated_socket_options(EmOpts, #socket_options{}), Trackers}, self(), CbInfo], + {ok, Pid} = tls_dyn_connection_sup:start_child(DynSup, receiver, ConnArgs), + ssl_gen_statem:socket_control(ConnectionCb, Socket, [Pid, Sender], Transport, Trackers) + catch + error:{badmatch, {error, _} = Error} -> + Error + end. + split_options(Opts) -> split_options(Opts, emulated_options(), [], []). split_options([], _, SocketOpts, EmuOpts) -> diff --git a/lib/ssl/src/tls_sup.erl b/lib/ssl/src/tls_sup.erl index a425ae31e2..cafc563943 100644 --- a/lib/ssl/src/tls_sup.erl +++ b/lib/ssl/src/tls_sup.erl @@ -43,14 +43,13 @@ start_link() -> %%% Supervisor callback %%%========================================================================= -init([]) -> - - TLSConnetionSup = tls_connection_child_spec(), - ServerInstanceSup = server_instance_child_spec(), - - {ok, {{one_for_one, 10, 3600}, [TLSConnetionSup, - ServerInstanceSup - ]}}. +init([]) -> + ChildSpecs = [tls_connection_child_spec(), server_instance_child_spec()], + SupFlags = #{strategy => one_for_one, + intensity => 10, + period => 3600 + }, + {ok, {SupFlags, ChildSpecs}}. %%-------------------------------------------------------------------- @@ -58,19 +57,19 @@ init([]) -> %%-------------------------------------------------------------------- tls_connection_child_spec() -> - Name = tls_connection, - StartFunc = {tls_connection_sup, start_link, []}, - Restart = permanent, - Shutdown = 4000, - Modules = [tls_connection_sup], - Type = supervisor, - {Name, StartFunc, Restart, Shutdown, Type, Modules}. + #{id => tls_connection_sup, + start => {tls_connection_sup, start_link, []}, + restart => permanent, + shutdown => 4000, + modules => [tls_connection_sup], + type => supervisor + }. server_instance_child_spec() -> - Name = tls_server_sup, - StartFunc = {tls_server_sup, start_link, []}, - Restart = permanent, - Shutdown = 4000, - Modules = [tls_server_sup], - Type = supervisor, - {Name, StartFunc, Restart, Shutdown, Type, Modules}. + #{id => tls_server_sup, + start => {tls_server_sup, start_link, []}, + restart => permanent, + shutdown => 4000, + modules => [tls_server_sup], + type => supervisor + }. diff --git a/lib/ssl/test/dtls_api_SUITE.erl b/lib/ssl/test/dtls_api_SUITE.erl index 572702af02..4117bec12e 100644 --- a/lib/ssl/test/dtls_api_SUITE.erl +++ b/lib/ssl/test/dtls_api_SUITE.erl @@ -51,9 +51,12 @@ dtls_listen_two_sockets_5/0, dtls_listen_two_sockets_5/1, dtls_listen_two_sockets_6/0, - dtls_listen_two_sockets_6/1 + dtls_listen_two_sockets_6/1, + client_restarts/0, client_restarts/1 ]). +-include_lib("ssl/src/ssl_internal.hrl"). + %%-------------------------------------------------------------------- %% Common Test interface functions ----------------------------------- %%-------------------------------------------------------------------- @@ -80,7 +83,8 @@ api_tests() -> dtls_listen_two_sockets_3, dtls_listen_two_sockets_4, dtls_listen_two_sockets_5, - dtls_listen_two_sockets_6 + dtls_listen_two_sockets_6, + client_restarts ]. init_per_suite(Config0) -> @@ -300,7 +304,6 @@ dtls_listen_two_sockets_6(_Config) when is_list(_Config) -> ssl:close(S1), ok. - replay_window() -> [{doc, "Whitebox test of replay window"}]. replay_window(_Config) -> @@ -347,6 +350,65 @@ bits_to_list(Bits, I, Acc) -> 0 -> bits_to_list(Bits bsr 1, I+1, Acc) end. +client_restarts() -> + [{doc, "Test re-connection "}]. + +client_restarts(Config) -> + ClientOpts = ssl_test_lib:ssl_options(client_rsa_opts, Config), + ServerOpts = ssl_test_lib:ssl_options(server_rsa_verify_opts, Config), + {ClientNode, ServerNode, Hostname} = ssl_test_lib:run_where(Config), + + Server = + ssl_test_lib:start_server([{node, ServerNode}, {port, 0}, + {from, self()}, + {mfa, {ssl_test_lib, no_result, []}}, + {options, ServerOpts}]), + Port = ssl_test_lib:inet_port(Server), + Client0 = ssl_test_lib:start_client([{node, ClientNode}, + {port, Port}, {host, Hostname}, + {mfa, {ssl_test_lib, no_result, []}}, + {from, self()}, + {options, [{reuse_sessions, save} | ClientOpts]}]), + ReConnect = %% Whitebox re-connect test + fun({sslsocket, {gen_udp,_,dtls_gen_connection}, [Pid]} = Socket, ssl) -> + ct:log("~p Client Socket: ~p ~n", [self(), Socket]), + {ok, {{Adress,CPort},UDPSocket}=IntSocket} = gen_statem:call(Pid, {downgrade, self()}), + true = is_port(UDPSocket), + ct:log("Info: ~p~n", [inet:info(UDPSocket)]), + + {ok, #config{transport_info = CbInfo, connection_cb = ConnectionCb, + ssl = SslOpts0}} = ssl:handle_options(ClientOpts, client, Adress), + SslOpts = {SslOpts0, #socket_options{}, undefined}, + + ct:sleep(250), + ct:log("Client second connect: ~p ~p~n", [Socket, CbInfo]), + Res = ssl_gen_statem:connect(ConnectionCb, Adress, CPort, IntSocket, SslOpts, self(), CbInfo, infinity), + {Res, Pid} + end, + + Client0 ! {apply, self(), ReConnect}, + receive + {apply_res, {Res, _Prev}} -> + ct:log("Apply res: ~p~n", [Res]), + ok; + Msg -> + ct:log("Unhandled: ~p~n", [Msg]), + ct:fail({wrong_msg, Msg}) + end, + + receive + Msg2 -> + ct:log("Unhandled: ~p~n", [Msg2]), + ct:fail({wrong_msg, Msg2}) + after 200 -> + ct:log("Nothing received~n", []) + end, + + ssl_test_lib:close(Server), + ssl_test_lib:close(Client0), + + ok. + %%-------------------------------------------------------------------- %% Internal functions ------------------------------------------------ %%-------------------------------------------------------------------- diff --git a/lib/ssl/test/openssl_client_cert_SUITE.erl b/lib/ssl/test/openssl_client_cert_SUITE.erl index ad00f2da1f..263628c3c4 100644 --- a/lib/ssl/test/openssl_client_cert_SUITE.erl +++ b/lib/ssl/test/openssl_client_cert_SUITE.erl @@ -82,7 +82,7 @@ groups() -> [ {openssl_client, [], protocol_groups()}, {'tlsv1.3', [], tls_1_3_protocol_groups()}, - {'tlsv1.2', [], pre_tls_1_3_protocol_groups()}, + {'tlsv1.2', [], pre_tls_1_3_protocol_groups() ++ [{group, rsa_pss_rsae}, {group, rsa_pss_pss}]}, {'tlsv1.1', [], pre_tls_1_3_protocol_groups()}, {'tlsv1', [], pre_tls_1_3_protocol_groups()}, {'dtlsv1.2', [], pre_tls_1_3_protocol_groups()}, @@ -92,8 +92,10 @@ groups() -> {dsa, [], all_version_tests()}, {rsa_1_3, [], all_version_tests() ++ tls_1_3_tests() ++ [unsupported_sign_algo_client_auth, unsupported_sign_algo_cert_client_auth]}, - {rsa_pss_rsae, [], all_version_tests() ++ tls_1_3_tests()}, - {rsa_pss_pss, [], all_version_tests() ++ tls_1_3_tests()}, + {rsa_pss_rsae, [], all_version_tests()}, + {rsa_pss_pss, [], all_version_tests()}, + {rsa_pss_rsae_1_3, [], all_version_tests() ++ tls_1_3_tests()}, + {rsa_pss_pss_1_3, [], all_version_tests() ++ tls_1_3_tests()}, {ecdsa_1_3, [], all_version_tests() ++ tls_1_3_tests()}, {eddsa_1_3, [], all_version_tests() ++ tls_1_3_tests()} ]. @@ -122,8 +124,8 @@ pre_tls_1_3_protocol_groups() -> tls_1_3_protocol_groups() -> [{group, rsa_1_3}, - {group, rsa_pss_rsae}, - {group, rsa_pss_pss}, + {group, rsa_pss_rsae_1_3}, + {group, rsa_pss_pss_1_3}, {group, ecdsa_1_3}, {group, eddsa_1_3} ]. @@ -198,23 +200,26 @@ init_per_group(Group, Config0) when Group == rsa; [] -> {skip, {no_sup, Group, Version}} end; -init_per_group(Alg, Config) when Alg == rsa_pss_rsae; - Alg == rsa_pss_pss -> +init_per_group(Alg, Config) when + Alg == rsa_pss_rsae; + Alg == rsa_pss_pss; + Alg == rsa_pss_rsae_1_3; + Alg == rsa_pss_pss_1_3 -> Supports = crypto:supports(), RSAOpts = proplists:get_value(rsa_opts, Supports), case lists:member(rsa_pkcs1_pss_padding, RSAOpts) andalso lists:member(rsa_pss_saltlen, RSAOpts) andalso lists:member(rsa_mgf1_md, RSAOpts) - andalso ssl_test_lib:is_sane_oppenssl_pss(Alg) + andalso ssl_test_lib:is_sane_oppenssl_pss(rsa_alg(Alg)) of true -> #{client_config := COpts, - server_config := SOpts} = ssl_test_lib:make_rsa_pss_pem(Alg, [], Config, ""), - [{cert_key_alg, Alg} | + server_config := SOpts} = ssl_test_lib:make_rsa_pss_pem(rsa_alg(Alg), [], Config, ""), + [{cert_key_alg, rsa_alg(Alg)} | lists:delete(cert_key_alg, - [{client_cert_opts, COpts}, - {server_cert_opts, SOpts} | + [{client_cert_opts, openssl_sig_algs(Alg) ++ COpts}, + {server_cert_opts, sig_algs(rsa_alg(Alg)) ++ SOpts} | lists:delete(server_cert_opts, lists:delete(client_cert_opts, Config))])]; false -> @@ -433,3 +438,28 @@ hello_retry_client_auth_empty_cert_rejected() -> ssl_cert_tests:hello_retry_client_auth_empty_cert_rejected(). hello_retry_client_auth_empty_cert_rejected(Config) -> ssl_cert_tests:hello_retry_client_auth_empty_cert_rejected(Config). + +rsa_alg(rsa_pss_rsae_1_3) -> + rsa_pss_rsae; +rsa_alg(rsa_pss_pss_1_3) -> + rsa_pss_pss; +rsa_alg(Atom) -> + Atom. + +sig_algs(rsa_pss_pss) -> + [{signature_algs, [rsa_pss_pss_sha512, + rsa_pss_pss_sha384, + rsa_pss_pss_sha256]}]; +sig_algs(rsa_pss_rsae) -> + [{signature_algs,[rsa_pss_rsae_sha512, + rsa_pss_rsae_sha384, + rsa_pss_rsae_sha256]}]. + +openssl_sig_algs(rsa_pss_pss) -> + [{sigalgs, "rsa_pss_pss_sha256"}]; +openssl_sig_algs(rsa_pss_rsae) -> + [{sigalgs,"rsa_pss_rsae_sha256"}]; +openssl_sig_algs(rsa_pss_pss_1_3) -> + [{sigalgs, "rsa_pss_rsae_sha512:rsa_pss_rsae_sha384:rsa_pss_pss_sha256"}]; +openssl_sig_algs(rsa_pss_rsae_1_3) -> + [{sigalgs,"rsa_pss_rsae_sha512:rsa_pss_rsae_sha384:rsa_pss_rsae_sha256"}]. diff --git a/lib/ssl/test/property_test/ssl_eqc_cipher_format.erl b/lib/ssl/test/property_test/ssl_eqc_cipher_format.erl index cf6ed755f7..11330b111b 100644 --- a/lib/ssl/test/property_test/ssl_eqc_cipher_format.erl +++ b/lib/ssl/test/property_test/ssl_eqc_cipher_format.erl @@ -51,6 +51,8 @@ -endif. -endif. +-include_lib("public_key/include/public_key.hrl"). + -define('TLS_v1.3', 'tlsv1.3'). -define('TLS_v1.2', 'tlsv1.2'). -define('TLS_v1.1', 'tlsv1.1'). @@ -105,6 +107,11 @@ prop_tls_anon_cipher_suite_openssl_name() -> end ). +prop_tls_signature_algs() -> + ?FORALL(SigAlg, ?LET(SigAlg, sig_alg(), SigAlg), + true = lists:member(ssl_cipher:signature_algorithm_to_scheme(SigAlg), sig_schemes()) + ). + %%-------------------------------------------------------------------- %% Generators ----------------------------------------------- %%-------------------------------------------------------------------- @@ -272,3 +279,55 @@ openssl_legacy_names() -> "SRP-AES-128-CBC-SHA", "SRP-AES-256-CBC-SHA" ]. + + +sig_alg() -> + oneof([#'SignatureAlgorithm'{algorithm = ?'id-RSASSA-PSS', + parameters = #'RSASSA-PSS-params'{ + maskGenAlgorithm = + #'MaskGenAlgorithm'{algorithm = ?'id-mgf1', + parameters = #'HashAlgorithm'{algorithm = ?'id-sha256'}}}}, + #'SignatureAlgorithm'{algorithm = ?'id-RSASSA-PSS', + parameters = #'RSASSA-PSS-params'{ + maskGenAlgorithm = + #'MaskGenAlgorithm'{algorithm = ?'id-mgf1', + parameters = #'HashAlgorithm'{algorithm = ?'id-sha384'}}}}, + + #'SignatureAlgorithm'{algorithm = ?'id-RSASSA-PSS', + parameters = #'RSASSA-PSS-params'{ + maskGenAlgorithm = + #'MaskGenAlgorithm'{algorithm = ?'id-mgf1', + parameters = #'HashAlgorithm'{algorithm = ?'id-sha512'}}}}, + #'SignatureAlgorithm'{algorithm = ?sha256WithRSAEncryption}, + #'SignatureAlgorithm'{algorithm = ?sha384WithRSAEncryption}, + #'SignatureAlgorithm'{algorithm = ?sha512WithRSAEncryption}, + #'SignatureAlgorithm'{algorithm = ?'ecdsa-with-SHA256'}, + #'SignatureAlgorithm'{algorithm = ?'ecdsa-with-SHA384'}, + #'SignatureAlgorithm'{algorithm = ?'ecdsa-with-SHA512'}, + #'SignatureAlgorithm'{algorithm = ?'sha-1WithRSAEncryption'}, + #'SignatureAlgorithm'{algorithm = ?'ecdsa-with-SHA1'}, + #'SignatureAlgorithm'{algorithm = ?'id-Ed25519'}, + #'SignatureAlgorithm'{algorithm = ?'id-Ed448'}, + #'SignatureAlgorithm'{algorithm = ?'rsaEncryption', + parameters = 'NULL'}, + #'SignatureAlgorithm'{algorithm = ?'rsaEncryption'}, + #'SignatureAlgorithm'{algorithm = ?'id-RSASSA-PSS'}]). + +sig_schemes() -> + [rsa_pss_pss_sha256, + rsa_pss_pss_sha384, + rsa_pss_pss_sha512, + rsa_pkcs1_sha256, + rsa_pkcs1_sha384, + rsa_pkcs1_sha512, + ecdsa_secp256r1_sha256, + ecdsa_secp384r1_sha384, + ecdsa_secp521r1_sha512, + rsa_pkcs1_sha1, + rsa_pkcs1_sha1, + ecdsa_sha1, + eddsa_ed25519, + eddsa_ed448, + rsa_pkcs1_sha1, + rsa_pss_rsae, + rsa_pss_pss]. diff --git a/lib/ssl/test/ssl_api_SUITE.erl b/lib/ssl/test/ssl_api_SUITE.erl index ec831cfb90..22b199aa70 100644 --- a/lib/ssl/test/ssl_api_SUITE.erl +++ b/lib/ssl/test/ssl_api_SUITE.erl @@ -171,7 +171,13 @@ invalid_options_tls13/0, invalid_options_tls13/1, cookie/0, - cookie/1 + cookie/1, + warn_verify_none/0, + warn_verify_none/1, + suppress_warn_verify_none/0, + suppress_warn_verify_none/1, + check_random_nonce/0, + check_random_nonce/1 ]). %% Apply export @@ -192,6 +198,8 @@ tls_close/1, no_recv_no_active/1, ssl_getstat/1, + log/2, + get_connection_information/3, %%TODO Keep? run_error_server/1, run_error_server_close/1, @@ -292,7 +300,10 @@ gen_api_tests() -> invalid_options, cb_info, log_alert, - getstat + getstat, + warn_verify_none, + suppress_warn_verify_none, + check_random_nonce ]. handshake_paus_tests() -> @@ -405,6 +416,10 @@ init_per_testcase(conf_signature_algs, Config) -> sha -> {skip, "Tests needs certs with sha256"} end; +init_per_testcase(check_random_nonce, Config) -> + ssl_test_lib:ct_log_supported_protocol_versions(Config), + ct:timetrap({seconds, 20}), + Config; init_per_testcase(_TestCase, Config) -> ssl_test_lib:ct_log_supported_protocol_versions(Config), ct:timetrap({seconds, 10}), @@ -2485,7 +2500,128 @@ cookie(Config) when is_list(Config) -> cookie_extension(Config, true), cookie_extension(Config, false). +warn_verify_none() -> + [{doc, "Test that verify_none default generates warning."}]. +warn_verify_none(Config) when is_list(Config)-> + ok = logger:add_handler(?MODULE,?MODULE,#{config=>self()}), + ClientOpts = ssl_test_lib:ssl_options(client_rsa_opts, Config), + ServerOpts = ssl_test_lib:ssl_options(server_rsa_opts, Config), + + {ClientNode, ServerNode, Hostname} = ssl_test_lib:run_where(Config), + + Server = ssl_test_lib:start_server([{node, ServerNode}, {port, 0}, + {from, self()}, + {options, ServerOpts}, + {mfa, {ssl_test_lib, no_result, []}}]), + Port = ssl_test_lib:inet_port(Server), + Client = ssl_test_lib:start_client([{node, ClientNode}, {port, Port}, + {from, self()}, + {host, Hostname}, + {options, ClientOpts}, + {mfa, {ssl_test_lib, no_result, []}}]), + receive + warning_generated -> + ok = logger:remove_handler(?MODULE) + after 500 -> + ok = logger:remove_handler(?MODULE), + ct:fail(no_warning) + end, + ssl_test_lib:close(Client), + ssl_test_lib:close(Server). + +suppress_warn_verify_none() -> + [{doc, "Test that explicit verify_none supresses warning."}]. +suppress_warn_verify_none(Config) when is_list(Config)-> + ok = logger:add_handler(?MODULE,?MODULE,#{config=>self()}), + + ClientOpts = ssl_test_lib:ssl_options(client_rsa_opts, Config), + ServerOpts = ssl_test_lib:ssl_options(server_rsa_opts, Config), + + {ClientNode, ServerNode, Hostname} = ssl_test_lib:run_where(Config), + + Server = ssl_test_lib:start_server([{node, ServerNode}, {port, 0}, + {from, self()}, + {options, ServerOpts}, + {mfa, {ssl_test_lib, no_result, []}}]), + Port = ssl_test_lib:inet_port(Server), + Client = ssl_test_lib:start_client([{node, ClientNode}, {port, Port}, + {from, self()}, + {host, Hostname}, + {options, [{verify, verify_none} | ClientOpts]}, + {mfa, {ssl_test_lib, no_result, []}}]), + receive + warning_generated -> + ok = logger:remove_handler(?MODULE), + ct:fail(warning) + after 500 -> + ok = logger:remove_handler(?MODULE) + end, + ssl_test_lib:close(Client), + ssl_test_lib:close(Server). + +%%-------------------------------------------------------------------- +check_random_nonce() -> + [{doc,"Test random nonce - expecting 32B random for TLS1.3 and 4B UTC " + "epoch with 28B random for older version"}]. +check_random_nonce(Config) when is_list(Config) -> + ClientOpts = ssl_test_lib:ssl_options(client_rsa_opts, Config), + ServerOpts = ssl_test_lib:ssl_options(server_rsa_opts, Config), + {ClientNode, ServerNode, Hostname} = ssl_test_lib:run_where(Config), + N = 10, + F = fun (Id) -> establish_connection(Id,ServerNode, ServerOpts, + ClientNode, ClientOpts, + Hostname) + end, + ConnectionPairs = [F(Id) || Id <- lists:seq(1, N)], + Randoms = lists:flatten([ssl_test_lib:get_result([Server, Client]) || + {Server, Client} <- ConnectionPairs]), + Deltas = [abs(FourBytes - SecsSince) || + {_Id, {_, <<FourBytes:32, _/binary>>}, SecsSince} <- Randoms], + MeanDelta = lists:sum(Deltas) div N, + case ?config(version, Config) of + 'tlsv1.3' -> + %% 4B "random" expected since TLS1.3 + RndThreshold = 10000, + true = MeanDelta > RndThreshold; + _ -> + %% 4 epoch based bytes expected pre TLS1.3 + EpochThreshold = 10, + true = MeanDelta < EpochThreshold + end, + [begin + ssl_test_lib:close(Server), + ssl_test_lib:close(Client) + end || {Server, Client} <- ConnectionPairs]. + +establish_connection(Id, ServerNode, ServerOpts, ClientNode, ClientOpts, Hostname) -> + Server = + ssl_test_lib:start_server([{node, ServerNode}, {port, 0}, + {from, self()}, + {mfa, {?MODULE, get_connection_information, + [Id, server_random]}}, + {options, [{verify, verify_peer} | ServerOpts]}]), + + ListenPort = ssl_test_lib:inet_port(Server), + Client = + ssl_test_lib:start_client([{node, ClientNode}, {port, ListenPort}, + {host, Hostname}, + {from, self()}, + {mfa, {?MODULE, get_connection_information, + [Id, client_random]}}, + {options, [{verify, verify_peer} |ClientOpts]}]), + + ct:log("Testcase ~p, Client ~p Server ~p ~n", + [self(), Client, Server]), + {Server, Client}. + %%% Checker functions +get_connection_information(Socket, ConnectionId, InfoType) -> + {ok, [ConnectionInfo]} = ssl:connection_information(Socket, [InfoType]), + {ConnectionId, ConnectionInfo, secs_since_1970()}. +secs_since_1970() -> + calendar:datetime_to_gregorian_seconds( + calendar:universal_time()) - 62167219200. + connection_information_result(Socket) -> {ok, Info = [_ | _]} = ssl:connection_information(Socket), case length(Info) > 3 of @@ -2909,3 +3045,8 @@ ssl_getstat(Socket) -> _ -> ok end. + +log(#{msg:={report,Report}},#{config:=Pid}) -> + Pid ! warning_generated; +log(_,_) -> + ok. diff --git a/lib/ssl/test/ssl_cert_SUITE.erl b/lib/ssl/test/ssl_cert_SUITE.erl index 42c44e4855..1eef4a48be 100644 --- a/lib/ssl/test/ssl_cert_SUITE.erl +++ b/lib/ssl/test/ssl_cert_SUITE.erl @@ -146,11 +146,11 @@ groups() -> {'tlsv1', [], ssl_protocol_groups()}, {'dtlsv1.2', [], tls_1_2_protocol_groups()}, {'dtlsv1', [], ssl_protocol_groups()}, - {rsa, [], all_version_tests() ++ rsa_tests() ++ pre_tls_1_3_rsa_tests()}, + {rsa, [], all_version_tests() ++ rsa_tests() ++ pre_tls_1_3_rsa_tests() ++ [client_auth_seelfsigned_peer]}, {ecdsa, [], all_version_tests()}, {dsa, [], all_version_tests()}, {rsa_1_3, [], all_version_tests() ++ rsa_tests() ++ - tls_1_3_tests() ++ tls_1_3_rsa_tests() ++ [basic_rsa_1024]}, + tls_1_3_tests() ++ tls_1_3_rsa_tests() ++ [client_auth_seelfsigned_peer, basic_rsa_1024]}, {rsa_pss_rsae, [], all_version_tests() ++ tls_1_2_rsa_tests()}, {rsa_pss_rsae_1_3, [], all_version_tests() ++ rsa_tests() ++ tls_1_3_tests() ++ tls_1_3_rsa_tests()}, {rsa_pss_pss, [], all_version_tests()}, @@ -226,7 +226,6 @@ all_version_tests() -> client_auth_do_not_allow_partial_chain, client_auth_partial_chain_fun_fail, client_auth_sni, - client_auth_seelfsigned_peer, missing_root_cert_no_auth, missing_root_cert_auth, missing_root_cert_auth_user_verify_fun_accept, @@ -296,37 +295,9 @@ do_init_per_group(Alg, Config) when Alg == rsa_pss_rsae; true -> #{client_config := COpts, server_config := SOpts} = ssl_test_lib:make_rsa_pss_pem(rsa_alg(Alg), [], Config, ""), - [{cert_key_alg, rsa_alg(Alg)}, - {extra_client, [{signature_algs, [rsa_pss_pss_sha512, - rsa_pss_pss_sha384, - rsa_pss_pss_sha256, - rsa_pss_rsae_sha512, - rsa_pss_rsae_sha384, - rsa_pss_rsae_sha256, - rsa_pkcs1_sha512, - rsa_pkcs1_sha384, - rsa_pkcs1_sha256, - rsa_pkcs1_sha1, - {sha512, rsa}, - {sha384, rsa}, - {sha256, rsa}, - {sha224, rsa} - ]}]}, - {extra_server, [{signature_algs, [rsa_pss_pss_sha512, - rsa_pss_pss_sha384, - rsa_pss_pss_sha256, - rsa_pss_rsae_sha512, - rsa_pss_rsae_sha384, - rsa_pss_rsae_sha256, - {sha512, ecdsa}, - {sha512, rsa}, - {sha384, ecdsa}, - {sha384, rsa}, - {sha256, ecdsa}, - {sha256, rsa}, - {sha224, ecdsa}, - {sha224, rsa} - ]}]} | + [{cert_key_alg, Alg}, + {extra_client, sig_algs(Alg)}, + {extra_server, sig_algs(Alg)} | lists:delete(cert_key_alg, [{client_cert_opts, COpts}, {server_cert_opts, SOpts} | @@ -1317,3 +1288,12 @@ chain_and_root(Config) -> {ok, ExtractedCAs} = ssl_pkix_db:extract_trusted_certs({der, proplists:get_value(cacerts, Config)}), {ok, Root, Chain} = ssl_certificate:certificate_chain(OwnCert, ets:new(foo, []), ExtractedCAs, [], encoded), {Chain, Root}. + +sig_algs(rsa_pss_pss) -> + [{signature_algs, [rsa_pss_pss_sha512, + rsa_pss_pss_sha384, + rsa_pss_pss_sha256]}]; +sig_algs(rsa_pss_rsae) -> + [{signature_algs, [rsa_pss_rsae_sha512, + rsa_pss_rsae_sha384, + rsa_pss_rsae_sha256]}]. diff --git a/lib/ssl/test/ssl_dist_SUITE.erl b/lib/ssl/test/ssl_dist_SUITE.erl index 67be91196f..5c8a8d9138 100644 --- a/lib/ssl/test/ssl_dist_SUITE.erl +++ b/lib/ssl/test/ssl_dist_SUITE.erl @@ -58,7 +58,9 @@ verify_fun_fail/0, verify_fun_fail/1, verify_fun_pass/0, - verify_fun_pass/1 + verify_fun_pass/1, + epmd_module/0, + epmd_module/1 ]). %% Apply export @@ -75,6 +77,12 @@ verify_pass_always/3, verify_fail_always/3]). +%% Epmd module export +-export([start_link/0, + register_node/2, + register_node/3, + port_please/2, + address_please/3]). -define(DEFAULT_TIMETRAP_SECS, 240). -define(AWAIT_SSL_NODE_UP_TIMEOUT, 30000). @@ -102,7 +110,8 @@ all() -> connect_options, use_interface, verify_fun_fail, - verify_fun_pass + verify_fun_pass, + epmd_module ]. init_per_suite(Config0) -> @@ -382,6 +391,53 @@ verify_fun_pass(Config) when is_list(Config) -> gen_dist_test(verify_fun_pass_test, [{tls_verify_opts, AddTLSVerifyOpts} | Config]). %%-------------------------------------------------------------------- +epmd_module() -> + [{doc,"Test that custom epmd_modules work"}]. +epmd_module(Config0) when is_list(Config0) -> + Config = [{hostname, "dummy"} | Config0], + NH1 = start_ssl_node(Config, "-epmd_module " ++ atom_to_list(?MODULE)), + NH2 = start_ssl_node(Config, "-epmd_module " ++ atom_to_list(?MODULE)), + + {ok, Port1} = apply_on_ssl_node(NH1, fun() -> application:get_env(kernel, dist_listen_port) end), + {ok, Port2} = apply_on_ssl_node(NH2, fun() -> application:get_env(kernel, dist_listen_port) end), + apply_on_ssl_node(NH1, fun() -> application:set_env(kernel, dist_connect_port, Port2) end), + apply_on_ssl_node(NH2, fun() -> application:set_env(kernel, dist_connect_port, Port1) end), + + try + basic_test(NH1, NH2, Config) + catch + _:Reason -> + stop_ssl_node(NH1), + stop_ssl_node(NH2), + ct:fail(Reason) + end, + stop_ssl_node(NH1), + stop_ssl_node(NH2), + success(Config). + +start_link() -> + ignore. + +register_node(Name, Port) -> + register_node(Name, Port, inet_tcp). +register_node(_Name, Port, _Driver) -> + %% Save the port number we're listening on. + application:set_env(kernel, dist_listen_port, Port), + Creation = rand:uniform(3), + {ok, Creation}. + +port_please(_Name, _Ip) -> + {ok, Port} = application:get_env(kernel, dist_connect_port), + {port, Port, 5}. + +address_please(_Name, "dummy", AddressFamily) -> + %% Use localhost. + {ok,Host} = inet:gethostname(), + inet:getaddr(Host, AddressFamily); +address_please(_, _, _) -> + {error, nxdomain}. + +%%-------------------------------------------------------------------- %%% Internal functions ----------------------------------------------- %%-------------------------------------------------------------------- gen_dist_test(Test, Config) -> @@ -653,11 +709,17 @@ start_ssl_node(Config, XArgs) -> mk_node_name(Config) -> N = erlang:unique_integer([positive]), Case = proplists:get_value(testcase, Config), + Hostname = + case proplists:get_value(hostname, Config) of + undefined -> ""; + Host -> "@" ++ Host + end, atom_to_list(?MODULE) ++ "_" ++ atom_to_list(Case) ++ "_" - ++ integer_to_list(N). + ++ integer_to_list(N) ++ Hostname. + setup_certs(Config) -> PrivDir = proplists:get_value(priv_dir, Config), diff --git a/lib/ssl/test/ssl_dist_test_lib.erl b/lib/ssl/test/ssl_dist_test_lib.erl index e85eddaf08..d0fda480b2 100644 --- a/lib/ssl/test/ssl_dist_test_lib.erl +++ b/lib/ssl/test/ssl_dist_test_lib.erl @@ -73,7 +73,9 @@ apply_on_ssl_node(Node, Ref, Msg) -> stop_ssl_node(#node_handle{connection_handler = Handler, socket = Socket, - name = Name}) -> + name = Name, + logfile = LogPath, + dumpfile = DumpPath}) -> test_server:format("Trying to stop ssl node ~s.~n", [Name]), Mon = erlang:monitor(process, Handler), unlink(Handler), @@ -93,13 +95,24 @@ stop_ssl_node(#node_handle{connection_handler = Handler, Error -> erlang:demonitor(Mon, [flush]), ct:pal("stop_ssl_node/1 ~s Warning ~p ~n", [Name,Error]) - end. + end, + case file:read_file(LogPath) of + {ok, Binary} -> + ct:pal("LogPath(~pB) = ~p~n~s", [filelib:file_size(LogPath), LogPath, + Binary]); + _ -> + ok + end, + ct:pal("DumpPath(~pB) = ~p~n", [filelib:file_size(DumpPath), DumpPath]). start_ssl_node(Name, Args) -> {ok, LSock} = gen_tcp:listen(0, [binary, {packet, 4}, {active, false}]), {ok, ListenPort} = inet:port(LSock), - CmdLine = mk_node_cmdline(ListenPort, Name, Args), + {ok, Pwd} = file:get_cwd(), + LogFilePath = filename:join([Pwd, "error_log." ++ Name]), + DumpFilePath = filename:join([Pwd, "erl_crash_dump." ++ Name]), + CmdLine = mk_node_cmdline(ListenPort, Name, Args, LogFilePath, DumpFilePath), test_server:format("Attempting to start ssl node ~ts: ~ts~n", [Name, CmdLine]), case open_port({spawn, CmdLine}, []) of Port when is_port(Port) -> @@ -108,8 +121,8 @@ start_ssl_node(Name, Args) -> case await_ssl_node_up(Name, LSock) of #node_handle{} = NodeHandle -> test_server:format("Ssl node ~s started.~n", [Name]), - NodeName = list_to_atom(Name ++ "@" ++ host_name()), - NodeHandle#node_handle{nodename = NodeName}; + NodeHandle#node_handle{logfile = LogFilePath, + dumpfile = DumpFilePath}; Error -> exit({failed_to_start_node, Name, Error}) end; @@ -123,7 +136,7 @@ host_name() -> %% atom_to_list(node())), Host. -mk_node_cmdline(ListenPort, Name, Args) -> +mk_node_cmdline(ListenPort, Name, Args, LogPath, DumpPath) -> Static = "-detached -noinput", Pa = filename:dirname(code:which(?MODULE)), Prog = case catch init:get_argument(progname) of @@ -134,7 +147,6 @@ mk_node_cmdline(ListenPort, Name, Args) -> false -> "-sname "; _ -> "-name " end, - {ok, Pwd} = file:get_cwd(), "\"" ++ Prog ++ "\" " ++ Static ++ " " ++ NameSw ++ " " ++ Name ++ " " @@ -145,10 +157,10 @@ mk_node_cmdline(ListenPort, Name, Args) -> ++ host_name() ++ " " ++ integer_to_list(ListenPort) ++ " " ++ Args ++ " " - ++ "-env ERL_CRASH_DUMP " ++ Pwd ++ "/erl_crash_dump." ++ Name ++ " " + ++ "-env ERL_CRASH_DUMP " ++ DumpPath ++ " " ++ "-kernel inet_dist_connect_options \"[{recbuf,12582912},{sndbuf,12582912},{high_watermark,8388608},{low_watermark,4194304}]\" " ++ "-kernel inet_dist_listen_options \"[{recbuf,12582912},{sndbuf,12582912},{high_watermark,8388608},{low_watermark,4194304}]\" " - ++ "-kernel error_logger \"{file,\\\"" ++ Pwd ++ "/error_log." ++ Name ++ "\\\"}\" " + ++ "-kernel error_logger \"{file,\\\"" ++ LogPath ++ "\\\"}\" " ++ "-setcookie " ++ atom_to_list(erlang:get_cookie()). %% @@ -173,31 +185,36 @@ await_ssl_node_up(Name, LSock) -> end. check_ssl_node_up(Socket, Name, Bin) -> + NodeName = + case string:find(Name,"@") of + nomatch -> + list_to_atom(Name++"@"++host_name()); + _ -> + list_to_atom(Name) + end, case catch binary_to_term(Bin) of {'EXIT', _} -> gen_tcp:close(Socket), exit({bad_data_received_from_ssl_node, Name, Bin}); {ssl_node_up, NodeName} -> - case list_to_atom(Name++"@"++host_name()) of - NodeName -> - Parent = self(), - Go = make_ref(), - %% Spawn connection handler on test server side - Pid = spawn( - fun () -> - link(group_leader()), - receive Go -> ok end, - process_flag(trap_exit, true), - tstsrvr_con_loop(Name, Socket, Parent) - end), - ok = gen_tcp:controlling_process(Socket, Pid), - Pid ! Go, - #node_handle{connection_handler = Pid, - socket = Socket, - name = Name}; - _ -> - exit({unexpected_ssl_node_connected, NodeName}) - end; + Parent = self(), + Go = make_ref(), + %% Spawn connection handler on test server side + Pid = spawn( + fun () -> + link(group_leader()), + receive Go -> ok end, + process_flag(trap_exit, true), + tstsrvr_con_loop(Name, Socket, Parent) + end), + ok = gen_tcp:controlling_process(Socket, Pid), + Pid ! Go, + #node_handle{connection_handler = Pid, + socket = Socket, + nodename = NodeName, + name = Name}; + {ssl_node_up, OtherNodeName} -> + exit({unexpected_ssl_node_connected, OtherNodeName}); Msg -> exit({unexpected_msg_instead_of_ssl_node_up, Name, Msg}) end. diff --git a/lib/ssl/test/ssl_dist_test_lib.hrl b/lib/ssl/test/ssl_dist_test_lib.hrl index 86b9b37026..9b1e1045ba 100644 --- a/lib/ssl/test/ssl_dist_test_lib.hrl +++ b/lib/ssl/test/ssl_dist_test_lib.hrl @@ -22,5 +22,7 @@ {connection_handler, socket, name, - nodename} + nodename, + logfile, + dumpfile} ). diff --git a/lib/ssl/test/ssl_eqc_SUITE.erl b/lib/ssl/test/ssl_eqc_SUITE.erl index 4bfff1585e..cf6185e591 100644 --- a/lib/ssl/test/ssl_eqc_SUITE.erl +++ b/lib/ssl/test/ssl_eqc_SUITE.erl @@ -36,6 +36,7 @@ tls_cipher_openssl_suite_names/1, tls_anon_cipher_suite_names/1, tls_anon_cipher_openssl_suite_names/1, + tls_signature_algs/1, tls_unorded_chains/1, tls_extraneous_chain/1, tls_extraneous_chains/1, @@ -54,6 +55,7 @@ all() -> tls_cipher_openssl_suite_names, tls_anon_cipher_suite_names, tls_anon_cipher_openssl_suite_names, + tls_signature_algs, tls_unorded_chains, tls_extraneous_chain, tls_extraneous_chains, @@ -101,6 +103,11 @@ tls_anon_cipher_openssl_suite_names(Config) when is_list(Config) -> true = ct_property_test:quickcheck(ssl_eqc_cipher_format:prop_tls_anon_cipher_suite_openssl_name(), Config). +tls_signature_algs(Config) when is_list(Config) -> + %% manual test: proper:quickcheck(ssl_eqc_handshake:prop_tls_signature_algs()). + true = ct_property_test:quickcheck(ssl_eqc_cipher_format:prop_tls_signature_algs(), + Config). + tls_unorded_chains(Config) when is_list(Config) -> %% manual test: proper:quickcheck(ssl_eqc_chain:prop_tls_ordered_path("/tmp") ssl:start(), diff --git a/lib/ssl/test/ssl_session_SUITE.erl b/lib/ssl/test/ssl_session_SUITE.erl index 8f2be84a84..c74052f6b4 100644 --- a/lib/ssl/test/ssl_session_SUITE.erl +++ b/lib/ssl/test/ssl_session_SUITE.erl @@ -159,6 +159,7 @@ init_per_testcase(server_max_session_table, Config) -> ct:timetrap({seconds, 30}), Config; init_per_testcase(_, Config) -> + ct:timetrap({seconds, 30}), Config. end_per_testcase(reuse_session_expired, Config) -> @@ -461,49 +462,49 @@ explicit_session_reuse_expired(Config) when is_list(Config) -> no_reuses_session_server_restart_new_cert() -> [{doc,"Check that a session is not reused if the server is restarted with a new cert."}]. no_reuses_session_server_restart_new_cert(Config) when is_list(Config) -> + ClientOpts = ssl_test_lib:ssl_options(client_rsa_der_opts, Config), + ServerOpts = ssl_test_lib:ssl_options(server_rsa_der_verify_opts, Config), + RSA1024ServerOpts = ssl_test_lib:ssl_options(server_rsa_1024_der_opts, Config), + RSA1024ClientOpts = ssl_test_lib:ssl_options(client_rsa_1024_der_opts, Config), - ClientOpts = ssl_test_lib:ssl_options(client_rsa_opts, Config), - ServerOpts = ssl_test_lib:ssl_options(server_rsa_verify_opts, Config), - RSA1024ServerOpts = ssl_test_lib:ssl_options(server_rsa_1024_opts, Config), - RSA1024ClientOpts = ssl_test_lib:ssl_options(client_rsa_1024_opts, Config), {ClientNode, ServerNode, Hostname} = ssl_test_lib:run_where(Config), - Server = + Server0 = ssl_test_lib:start_server([{node, ServerNode}, {port, 0}, {from, self()}, - {mfa, {ssl_test_lib, session_info_result, []}}, + {mfa, {ssl_test_lib, session_info_result, []}}, {options, ServerOpts}]), - Port = ssl_test_lib:inet_port(Server), + Port = ssl_test_lib:inet_port(Server0), Client0 = ssl_test_lib:start_client([{node, ClientNode}, - {port, Port}, {host, Hostname}, - {mfa, {ssl_test_lib, no_result, []}}, - {from, self()}, {options, [{reuse_sessions, save} | ClientOpts]}]), - SessionInfo = - receive - {Server, Info} -> - Info - end, + {port, Port}, {host, Hostname}, + {mfa, {ssl_test_lib, session_info_result, []}}, + {from, self()}, {options, [{reuse_sessions, save} | ClientOpts]}]), + Info0 = receive {Server0, Info00} -> Info00 end, + Info0 = receive {Client0, Info01} -> Info01 end, - ssl_test_lib:close(Server), + ct:sleep(?SLEEP), ssl_test_lib:close(Client0), + ssl_test_lib:close(Server0), - Server1 = - ssl_test_lib:start_server([{node, ServerNode}, {port, Port}, - {from, self()}, - {mfa, {ssl_test_lib, no_result, []}}, - {options, [{reuseaddr, true} | RSA1024ServerOpts]}]), + Server1 = ssl_test_lib:start_server([{node, ServerNode}, {port, Port}, + {from, self()}, + {mfa, {ssl_test_lib, session_info_result, []}}, + {options, [{reuseaddr, true} | RSA1024ServerOpts]}]), + + Client1 = ssl_test_lib:start_client([{node, ClientNode}, + {port, Port}, {host, Hostname}, + {mfa, {ssl_test_lib, session_info_result, []}}, + {from, self()}, {options, RSA1024ClientOpts}]), + Info1 = receive {Server1, Info10} -> Info10 end, - Client1 = - ssl_test_lib:start_client([{node, ClientNode}, - {port, Port}, {host, Hostname}, - {mfa, {ssl_test_lib, session_info_result, []}}, - {from, self()}, {options, RSA1024ClientOpts}]), receive - {Client1, SessionInfo} -> + {Client1, Info0} -> ct:fail(session_reused_when_server_has_new_cert); - {Client1, _Other} -> - ok + {Client1, Info1} -> + ct:pal("First: ~p~nSecond ~p~n",[Info0, Info1]); + Unexpected -> + ct:fail({unexpected, Unexpected, {Client1, Info1}}) end, ssl_test_lib:close(Server1), ssl_test_lib:close(Client1). diff --git a/lib/ssl/test/ssl_test_lib.erl b/lib/ssl/test/ssl_test_lib.erl index e4c23c22cf..8b30819816 100644 --- a/lib/ssl/test/ssl_test_lib.erl +++ b/lib/ssl/test/ssl_test_lib.erl @@ -108,6 +108,7 @@ check_ok/1, check_result/4, check_result/2, + get_result/1, gen_check_result/4, basic_alert/4, session_id/1, @@ -409,7 +410,7 @@ run_where(_, ipv6) -> {ClientNode, ServerNode, Host}. node_to_hostip(Node, Role) -> - [_ , Host] = string:tokens(atom_to_list(Node), "@"), + Host = rpc:call(Node, net_adm, localhost, []), {ok, Address} = inet:getaddr(Host, inet), %% Convert client addresses in 127.0.0.0/24 subnet to the atom 'localhost'. %% This is a workaround for testcase problems caused by the fact that @@ -1041,7 +1042,15 @@ client_loop_core(Socket, Pid, Transport) -> {ssl_closed, Socket} -> ok; {gen_tcp, closed} -> - ok + ok; + {apply, From, Fun} -> + try + Res = Fun(Socket, Transport), + From ! {apply_res, Res} + catch E:R:ST -> + From ! {apply_res, {E,R,ST}} + end, + client_loop_core(Socket, Pid, Transport) end. client_cont_loop(_Node, Host, Port, Pid, Transport, Options, cancel, _Opts) -> @@ -1113,9 +1122,20 @@ close(Pid, Timeout) -> exit(Pid, kill) end. +get_result(Pids) -> + get_result(Pids, []). + +get_result([], Acc) -> + Acc; +get_result([Pid | Tail], Acc) -> + receive + {Pid, Msg} -> + get_result(Tail, [Msg | Acc]) + end. + check_result(Server, ServerMsg, Client, ClientMsg) -> {ClientIP, ClientPort} = get_ip_port(ServerMsg), - receive + receive {Server, ServerMsg} -> check_result(Client, ClientMsg); %% Workaround to accept local addresses (127.0.0.0/24) @@ -1710,16 +1730,20 @@ make_rsa_cert(Config) -> ServerChain = proplists:get_value(server_chain, Config, default_cert_chain_conf()), CertChainConf = gen_conf(rsa, rsa, ClientChain, ServerChain), GenCertData = public_key:pkix_test_data(CertChainConf), + #{client_config := ClientDerConf, server_config := ServerDerConf} = GenCertData, + [{server_config, ServerConf}, {client_config, ClientConf}] = x509_test:gen_pem_config_files(GenCertData, ClientFileBase, ServerFileBase), [{server_rsa_opts, [{reuseaddr, true} | ServerConf]}, - - {server_rsa_verify_opts, [{reuseaddr, true}, - {verify, verify_peer} | ServerConf]}, + {server_rsa_verify_opts, [{reuseaddr, true}, {verify, verify_peer} | ServerConf]}, {client_rsa_opts, ClientConf}, - {client_rsa_verify_opts, [{verify, verify_peer} |ClientConf]} - | Config]; + {client_rsa_verify_opts, [{verify, verify_peer} |ClientConf]}, + {server_rsa_der_opts, [{reuseaddr, true} | ServerDerConf]}, + {server_rsa_der_verify_opts, [{reuseaddr, true}, {verify, verify_peer} | ServerDerConf]}, + {client_rsa_der_opts, ClientDerConf}, + {client_rsa_der_verify_opts, [{verify, verify_peer} |ClientDerConf]} + | Config]; false -> Config end. @@ -1734,15 +1758,18 @@ make_rsa_1024_cert(Config) -> ServerChain = proplists:get_value(server_chain, Config, default_cert_chain_conf()), CertChainConf = gen_conf('rsa-1024', 'rsa-1024', ClientChain, ServerChain), GenCertData = public_key:pkix_test_data(CertChainConf), + #{client_config := ClientDerConf, server_config := ServerDerConf} = GenCertData, [{server_config, ServerConf}, {client_config, ClientConf}] = x509_test:gen_pem_config_files(GenCertData, ClientFileBase, ServerFileBase), [{server_rsa_1024_opts, [{ssl_imp, new},{reuseaddr, true} | ServerConf]}, - - {server_rsa_1024_verify_opts, [{ssl_imp, new}, {reuseaddr, true}, - {verify, verify_peer} | ServerConf]}, + {server_rsa_1024_verify_opts, [{ssl_imp, new}, {reuseaddr, true}, {verify, verify_peer} | ServerConf]}, {client_rsa_1024_opts, ClientConf}, - {client_rsa_1024_verify_opts, [{verify, verify_peer} |ClientConf]} + {client_rsa_1024_verify_opts, [{verify, verify_peer} |ClientConf]}, + {server_rsa_1024_der_opts, [{ssl_imp, new},{reuseaddr, true} | ServerDerConf]}, + {server_rsa_1024_der_verify_opts, [{ssl_imp, new}, {reuseaddr, true}, {verify, verify_peer} | ServerDerConf]}, + {client_rsa_1024_der_opts, ClientDerConf}, + {client_rsa_1024_der_verify_opts, [{verify, verify_peer} |ClientDerConf]} | Config]; false -> Config @@ -2080,6 +2107,7 @@ start_client(openssl, Port, ClientOpts, Config) -> Ciphers = proplists:get_value(ciphers, ClientOpts, ssl:cipher_suites(default,Version)), Groups0 = proplists:get_value(groups, ClientOpts), CertArgs = openssl_cert_options(ClientOpts, client), + SigAlgs = openssl_sigalgs(proplists:get_value(sigalgs, ClientOpts, undefined)), AlpnArgs = openssl_alpn_options(proplists:get_value(alpn, ClientOpts, undefined)), NpnArgs = openssl_npn_options(proplists:get_value(np, ClientOpts, undefined)), Reconnect = openssl_reconect_option(proplists:get_value(reconnect, ClientOpts, false)), @@ -2096,7 +2124,7 @@ start_client(openssl, Port, ClientOpts, Config) -> "-connect", hostname_format(HostName) ++ ":" ++ integer_to_list(Port), cipher_flag(Version), ciphers(Ciphers, Version), version_flag(Version)] - ++ CertArgs ++ AlpnArgs ++ NpnArgs ++ Reconnect ++ MaxFragLen ++ SessionArgs + ++ CertArgs ++ SigAlgs ++ AlpnArgs ++ NpnArgs ++ Reconnect ++ MaxFragLen ++ SessionArgs ++ Debug; Group -> ["s_client", @@ -2104,7 +2132,7 @@ start_client(openssl, Port, ClientOpts, Config) -> "-connect", hostname_format(HostName) ++ ":" ++ integer_to_list(Port), cipher_flag(Version), ciphers(Ciphers, Version), "-groups", Group, version_flag(Version)] - ++ CertArgs ++ AlpnArgs ++ NpnArgs ++ Reconnect ++ MaxFragLen ++ SessionArgs + ++ CertArgs ++ SigAlgs ++ AlpnArgs ++ NpnArgs ++ Reconnect ++ MaxFragLen ++ SessionArgs ++ Debug end, Args = maybe_force_ipv4(Args0), @@ -2347,6 +2375,11 @@ cert_option("-cert_chain", Value) -> cert_option(Opt, Value) -> [Opt, Value]. +openssl_sigalgs(undefined) -> + []; +openssl_sigalgs(SigAlgs) -> + ["-sigalgs", SigAlgs]. + supported_eccs(Opts) -> ToCheck = proplists:get_value(eccs, Opts, []), Supported = ssl:eccs(), diff --git a/lib/ssl/test/tls_1_3_record_SUITE.erl b/lib/ssl/test/tls_1_3_record_SUITE.erl index ab6d7bf33b..7b774ea1b9 100644 --- a/lib/ssl/test/tls_1_3_record_SUITE.erl +++ b/lib/ssl/test/tls_1_3_record_SUITE.erl @@ -36,7 +36,7 @@ -export([encode_decode/0, encode_decode/1, finished_verify_data/0, - finished_verify_data/1, + finished_verify_data/1, '1_RTT_handshake'/0, '1_RTT_handshake'/1, '0_RTT_handshake'/0, diff --git a/lib/ssl/test/tls_api_SUITE.erl b/lib/ssl/test/tls_api_SUITE.erl index 453ccf12ee..a5f4fe3709 100644 --- a/lib/ssl/test/tls_api_SUITE.erl +++ b/lib/ssl/test/tls_api_SUITE.erl @@ -512,8 +512,7 @@ tls_monitor_listener(Config) when is_list(Config) -> {options, tls_monitor_listen_opts(Version, ServerOpts)}]), _Port2 = ssl_test_lib:inet_port(Server2), - ProlistCounts1 = supervisor:count_children(ssl_listen_tracker_sup), - 2 = proplists:get_value(workers, ProlistCounts1), + 2 = count_children(workers, ssl_listen_tracker_sup), Sessions = session_info(Version), @@ -534,12 +533,11 @@ tls_monitor_listener(Config) when is_list(Config) -> ct:sleep(1000) end, - ProlistCounts2 = supervisor:count_children(ssl_listen_tracker_sup), Sessions1 = session_info(Version), true = (Sessions1 == 1), - - 1 = proplists:get_value(workers, ProlistCounts2). + + 1 = count_children(workers, ssl_listen_tracker_sup). %%-------------------------------------------------------------------- tls_tcp_msg() -> @@ -826,7 +824,7 @@ transport_close_in_inital_hello(Config) when is_list(Config) -> Sup = (whereis(tls_connection_sup)), - check_connection_workers(Sup, 2), + check_connection_processes(Sup, 2), Acceptor ! die, @@ -838,20 +836,20 @@ transport_close_in_inital_hello(Config) when is_list(Config) -> {'EXIT', Connector, _} -> ok end, - check_connection_workers(Sup, 0). + check_connection_processes(Sup, 0). -check_connection_workers(Sup, N) -> - check_connection_workers(Sup, N, 5). +check_connection_processes(Sup, N) -> + check_connection_processes(Sup, N, 5). -check_connection_workers(Sup, N, 0) -> - N = proplists:get_value(workers, supervisor:count_children(Sup)); -check_connection_workers(Sup, N, M) -> - case proplists:get_value(workers, supervisor:count_children(Sup)) of +check_connection_processes(Sup, N, 0) -> + N = count_children(supervisors, Sup); +check_connection_processes(Sup, N, M) -> + case count_children(supervisors, Sup) of N -> ok; _ -> ct:sleep(500), - check_connection_workers(Sup, N, M-1) + check_connection_processes(Sup, N, M-1) end. %%-------------------------------------------------------------------- @@ -1100,8 +1098,9 @@ tls_monitor_client_opts(_, Opts) -> Opts. session_info('tlsv1.3') -> - ProlistCounts = supervisor:count_children(tls_server_session_ticket_sup), - proplists:get_value(workers, ProlistCounts); + count_children(workers, tls_server_session_ticket_sup); session_info(_) -> - ProlistCounts = supervisor:count_children(ssl_server_session_cache_sup), - proplists:get_value(workers, ProlistCounts). + count_children(workers, ssl_server_session_cache_sup). + +count_children(ChildType, SupRef) -> + proplists:get_value(ChildType, supervisor:count_children(SupRef)). diff --git a/lib/ssl/vsn.mk b/lib/ssl/vsn.mk index 8f1847a5be..4064eb55f6 100644 --- a/lib/ssl/vsn.mk +++ b/lib/ssl/vsn.mk @@ -1 +1 @@ -SSL_VSN = 10.5.1 +SSL_VSN = 10.5.3 diff --git a/lib/stdlib/doc/src/gen_statem.xml b/lib/stdlib/doc/src/gen_statem.xml index c6790c1d48..2d64680412 100644 --- a/lib/stdlib/doc/src/gen_statem.xml +++ b/lib/stdlib/doc/src/gen_statem.xml @@ -1452,7 +1452,8 @@ handle_event(_, _, State, Data) -> </desc> </datatype> <datatype> - <name name="init_result"/> + <name name="init_result" n_vars="1"/> + <name name="init_result" n_vars="2"/> <desc> <p> For a succesful initialization, @@ -1479,7 +1480,8 @@ handle_event(_, _, State, Data) -> </desc> </datatype> <datatype> - <name name="state_enter_result"/> + <name name="state_enter_result" n_vars="1"/> + <name name="state_enter_result" n_vars="2"/> <desc> <p> <c><anno>State</anno></c> is the current state @@ -1502,7 +1504,8 @@ handle_event(_, _, State, Data) -> </desc> </datatype> <datatype> - <name name="event_handler_result"/> + <name name="event_handler_result" n_vars="1"/> + <name name="event_handler_result" n_vars="2"/> <desc> <p> <c><anno>StateType</anno></c> is @@ -1532,7 +1535,8 @@ handle_event(_, _, State, Data) -> </desc> </datatype> <datatype> - <name name="state_callback_result"/> + <name name="state_callback_result" n_vars="1"/> + <name name="state_callback_result" n_vars="2"/> <desc> <p> <c><anno>ActionType</anno></c> is diff --git a/lib/stdlib/doc/src/maps.xml b/lib/stdlib/doc/src/maps.xml index fdca524153..7c4fb9eaec 100644 --- a/lib/stdlib/doc/src/maps.xml +++ b/lib/stdlib/doc/src/maps.xml @@ -92,7 +92,7 @@ <p>The call fails with a <c>{badmap,Map}</c> exception if <c><anno>MapOrIter</anno></c> is not a map or valid iterator, or with <c>badarg</c> if <c><anno>Fun</anno></c> is not a - function of arity 3.</p> + function of arity 2.</p> <p><em>Example:</em></p> <code type="none"> > Fun = fun(K,V) when is_atom(K) -> {true, V*2}; (_,V) -> (V rem 2) =:= 0 end, diff --git a/lib/stdlib/doc/src/supervisor_bridge.xml b/lib/stdlib/doc/src/supervisor_bridge.xml index 97d0ae292a..97dac7cc07 100644 --- a/lib/stdlib/doc/src/supervisor_bridge.xml +++ b/lib/stdlib/doc/src/supervisor_bridge.xml @@ -72,16 +72,16 @@ <c><anno>Name</anno></c> using <c>register/2</c>.</p> </item> <item> - <p>If <c><anno>SupBridgeName</anno>={global,<anno>Name</anno>}</c>, + <p>If <c><anno>SupBridgeName</anno>={global,<anno>GlobalName</anno>}</c>, the supervisor bridge is registered globally as - <c><anno>Name</anno></c> using + <c><anno>GlobalName</anno></c> using <seemfa marker="kernel:global#register_name/2"> <c>global:register_name/2</c></seemfa>.</p> </item> <item> <p>If - <c><anno>SupBridgeName</anno>={via,<anno>Module</anno>,<anno>Name</anno>}</c>, - the supervisor bridge is registered as <c><anno>Name</anno></c> + <c><anno>SupBridgeName</anno>={via,<anno>Module</anno>,<anno>ViaName</anno>}</c>, + the supervisor bridge is registered as <c><anno>ViaName</anno></c> using a registry represented by <anno>Module</anno>. The <c>Module</c> callback is to export functions <c>register_name/2</c>, <c>unregister_name/1</c>, and <c>send/2</c>, diff --git a/lib/stdlib/src/erl_lint.erl b/lib/stdlib/src/erl_lint.erl index 7cafc9ce73..2eaf3d3d4a 100644 --- a/lib/stdlib/src/erl_lint.erl +++ b/lib/stdlib/src/erl_lint.erl @@ -2895,7 +2895,8 @@ type_def(Attr, Anno, TypeName, ProtoType, Args, St0) -> CheckType = {type, nowarn(), product, [ProtoType|Args]}, check_type(CheckType, St#lint{types=NewDefs}) end, - case is_default_type(TypePair) of + case is_default_type(TypePair) andalso + not member(no_auto_import_types, St0#lint.compile) of true -> case is_obsolete_builtin_type(TypePair) of true -> StoreType(St0); diff --git a/lib/stdlib/src/erl_stdlib_errors.erl b/lib/stdlib/src/erl_stdlib_errors.erl index 92518d058f..f9b1eb2126 100644 --- a/lib/stdlib/src/erl_stdlib_errors.erl +++ b/lib/stdlib/src/erl_stdlib_errors.erl @@ -296,7 +296,9 @@ format_re_error(compile, [_], _) -> [not_iodata]; format_re_error(compile, [Re, _Options], Cause) -> ReError = try re:compile(Re) of - _ -> [] + {ok, _} -> []; + {error, Reason} -> + {bad_regexp, Reason} catch _:_ -> not_iodata end, @@ -957,10 +959,21 @@ must_be_position(Pos) when is_integer(Pos) -> range; must_be_position(_) -> not_integer. must_be_regexp(Term) -> - try re:run("", Term) of - _ -> [] - catch - error:_ -> not_regexp + %% First check if we can compile the regexp as this + %% returns better error messages + try re:compile(Term) of + {ok, _} -> + []; + {error, Reason} -> + {bad_regexp, Reason} + catch error:_ -> + %% Then check if we can run it as this also allows + %% compiled reg exps + try re:run("", Term) of + _ -> [] + catch + error:_ -> not_regexp + end end. expand_error(already_owner) -> @@ -1052,6 +1065,11 @@ expand_error(not_pid) -> <<"not a pid">>; expand_error(not_regexp) -> <<"neither an iodata term nor a compiled regular expression">>; +expand_error({bad_regexp, {Reason,Column}}) -> + unicode:characters_to_binary( + io_lib:format("could not parse regular expression~n" + "~ts on character ~p", + [Reason, Column])); expand_error(not_tuple) -> <<"not a tuple">>; expand_error(not_tuple_or_list) -> diff --git a/lib/stdlib/src/gen_statem.erl b/lib/stdlib/src/gen_statem.erl index e91a24fd53..e206a826ea 100644 --- a/lib/stdlib/src/gen_statem.erl +++ b/lib/stdlib/src/gen_statem.erl @@ -63,8 +63,11 @@ from/0, callback_mode_result/0, init_result/1, + init_result/2, state_enter_result/1, + state_enter_result/2, event_handler_result/1, + event_handler_result/2, reply_action/0, enter_action/0, action/0 @@ -204,9 +207,10 @@ {'reply', % Reply to a caller From :: from(), Reply :: term()}. --type init_result(StateType) :: - {ok, State :: StateType, Data :: data()} | - {ok, State :: StateType, Data :: data(), +-type init_result(StateType) :: init_result(StateType, term()). +-type init_result(StateType, DataType) :: + {ok, State :: StateType, Data :: DataType} | + {ok, State :: StateType, Data :: DataType, Actions :: [action()] | action()} | 'ignore' | {'stop', Reason :: term()}. @@ -217,38 +221,43 @@ -type handle_event_result() :: event_handler_result(state()). %% --type state_enter_result(State) :: - {'next_state', % {next_state,NextState,NewData,[]} +-type state_enter_result(State) :: state_enter_result(State, term()). +-type state_enter_result(State, DataType) :: + {'next_state', % {next_state,State,NewData,[]} State, - NewData :: data()} | - {'next_state', % State transition, maybe to the same state + NewData :: DataType} | + {'next_state', % State entry for state State State, - NewData :: data(), + NewData :: DataType, Actions :: [enter_action()] | enter_action()} | state_callback_result(enter_action()). -type event_handler_result(StateType) :: + event_handler_result(StateType, term()). +-type event_handler_result(StateType, DataType) :: {'next_state', % {next_state,NextState,NewData,[]} NextState :: StateType, - NewData :: data()} | + NewData :: DataType} | {'next_state', % State transition, maybe to the same state NextState :: StateType, - NewData :: data(), + NewData :: DataType, Actions :: [action()] | action()} | state_callback_result(action()). -type state_callback_result(ActionType) :: + state_callback_result(ActionType, term()). +-type state_callback_result(ActionType, DataType) :: {'keep_state', % {keep_state,NewData,[]} - NewData :: data()} | + NewData :: DataType} | {'keep_state', % Keep state, change data - NewData :: data(), + NewData :: DataType, Actions :: [ActionType] | ActionType} | 'keep_state_and_data' | % {keep_state_and_data,[]} {'keep_state_and_data', % Keep state and data -> only actions Actions :: [ActionType] | ActionType} | %% {'repeat_state', % {repeat_state,NewData,[]} - NewData :: data()} | + NewData :: DataType} | {'repeat_state', % Repeat state, change data - NewData :: data(), + NewData :: DataType, Actions :: [ActionType] | ActionType} | 'repeat_state_and_data' | % {repeat_state_and_data,[]} {'repeat_state_and_data', % Repeat state and data -> only actions @@ -259,7 +268,7 @@ Reason :: term()} | {'stop', % Stop the server Reason :: term(), - NewData :: data()} | + NewData :: DataType} | %% {'stop_and_reply', % Reply then stop the server Reason :: term(), @@ -267,7 +276,7 @@ {'stop_and_reply', % Reply then stop the server Reason :: term(), Replies :: [reply_action()] | reply_action(), - NewData :: data()}. + NewData :: DataType}. -type request_id() :: term(). diff --git a/lib/stdlib/src/supervisor.erl b/lib/stdlib/src/supervisor.erl index 0d8b5a5ef4..7208068f4e 100644 --- a/lib/stdlib/src/supervisor.erl +++ b/lib/stdlib/src/supervisor.erl @@ -926,6 +926,8 @@ shutdown(#child{pid=Pid, shutdown=brutal_kill} = Child) -> case unlink_flush(Pid, Reason0) of killed -> ok; + shutdown when not (?is_permanent(Child)) -> + ok; {shutdown, _} when not (?is_permanent(Child)) -> ok; normal when not (?is_permanent(Child)) -> @@ -1041,6 +1043,10 @@ wait_dynamic_children(#child{shutdown=brutal_kill} = Child, Pids, Sz, wait_dynamic_children(Child, maps:remove({Pid, Mon}, Pids), Sz-1, TRef, EStack); + shutdown when not (?is_permanent(Child)) -> + wait_dynamic_children(Child, maps:remove({Pid, Mon}, Pids), + Sz-1, TRef, EStack); + {shutdown, _} when not (?is_permanent(Child)) -> wait_dynamic_children(Child, maps:remove({Pid, Mon}, Pids), Sz-1, TRef, EStack); diff --git a/lib/stdlib/src/supervisor_bridge.erl b/lib/stdlib/src/supervisor_bridge.erl index abbfb404a5..b796af30dc 100644 --- a/lib/stdlib/src/supervisor_bridge.erl +++ b/lib/stdlib/src/supervisor_bridge.erl @@ -64,8 +64,11 @@ start_link(Mod, StartArgs) -> gen_server:start_link(supervisor_bridge, [Mod, StartArgs, self], []). -spec start_link(SupBridgeName, Module, Args) -> Result when - SupBridgeName :: {local, Name} | {global, Name}, + SupBridgeName :: {local, Name} | {global, GlobalName} | + {via, Module, ViaName}, Name :: atom(), + GlobalName :: term(), + ViaName :: term(), Module :: module(), Args :: term(), Result :: {ok, Pid} | ignore | {error, Error}, diff --git a/lib/stdlib/test/re_SUITE.erl b/lib/stdlib/test/re_SUITE.erl index 5eef8ae48d..8e154cd829 100644 --- a/lib/stdlib/test/re_SUITE.erl +++ b/lib/stdlib/test/re_SUITE.erl @@ -1003,7 +1003,10 @@ bad_utf8_subject(Config) when is_list(Config) -> error_info(_Config) -> BadRegexp = {re_pattern,0,0,0,<<"xyz">>}, + BadErr = "neither an iodata term", {ok,GoodRegexp} = re:compile(".*"), + InvalidRegexp = <<"(.*))">>, + InvalidErr = "could not parse regular expression\n.*unmatched parentheses.*", L = [{compile, [not_iodata]}, {compile, [not_iodata, not_list],[{1,".*"},{2,".*"}]}, @@ -1021,28 +1024,35 @@ error_info(_Config) -> {internal_run, 4}, %Internal. {replace, [{a,b}, {x,y}, {z,z}],[{1,".*"},{2,".*"},{3,".*"}]}, - {replace, [{a,b}, BadRegexp, {z,z}],[{1,".*"},{2,".*"},{3,".*"}]}, + {replace, [{a,b}, BadRegexp, {z,z}],[{1,".*"},{2,BadErr},{3,".*"}]}, + {replace, [{a,b}, InvalidRegexp, {z,z}],[{1,".*"},{2,InvalidErr},{3,".*"}]}, {replace, [{a,b}, {x,y}, {z,z}, [a|b]],[{1,".*"},{2,".*"},{3,".*"},{4,".*"}]}, - {replace, [{a,b}, BadRegexp, [bad_option]],[{1,".*"},{2,".*"},{3,".*"}]}, + {replace, [{a,b}, BadRegexp, [bad_option]],[{1,".*"},{2,BadErr},{3,".*"}]}, + {replace, [{a,b}, InvalidRegexp, [bad_option]],[{1,".*"},{2,InvalidErr},{3,".*"}]}, {replace, ["", "", {z,z}, not_a_list],[{3,".*"},{4,".*"}]}, {run, [{a,b}, {x,y}],[{1,".*"},{2,".*"}]}, {run, [{a,b}, ".*"]}, {run, ["abc", {x,y}]}, - {run, ["abc", BadRegexp]}, + {run, ["abc", BadRegexp],[{2,BadErr}]}, + {run, ["abc", InvalidRegexp],[{2,InvalidErr}]}, {run, [{a,b}, {x,y}, []],[{1,".*"},{2,".*"}]}, - {run, ["abc", BadRegexp, []]}, + {run, ["abc", BadRegexp, []],[{2,BadErr}]}, + {run, ["abc", InvalidRegexp, []],[{2,InvalidErr}]}, {run, [{a,b}, {x,y}, [a|b]],[{1,".*"},{2,".*"},{3,".*"}]}, {run, [{a,b}, ".*", bad_options],[{1,".*"},{3,".*"}]}, {run, ["abc", {x,y}, [bad_option]],[{2,".*"},{3,".*"}]}, - {run, ["abc", BadRegexp, 9999],[{2,".*"},{3,".*"}]}, + {run, ["abc", BadRegexp, 9999],[{2,BadErr},{3,".*"}]}, + {run, ["abc", InvalidRegexp, 9999],[{2,InvalidErr},{3,".*"}]}, - {split, ["abc", BadRegexp]}, + {split, ["abc", BadRegexp],[{2,BadErr}]}, + {split, ["abc", InvalidRegexp],[{2,InvalidErr}]}, {split, [{a,b}, ".*"]}, - {split, ["abc", BadRegexp, [a|b]],[{2,".*"},{3,".*"}]}, + {split, ["abc", BadRegexp, [a|b]],[{2,BadErr},{3,".*"}]}, + {split, ["abc", InvalidRegexp, [a|b]],[{2,InvalidErr},{3,".*"}]}, {split, [{a,b}, ".*", [bad_option]]}, {ucompile, 2}, %Internal. diff --git a/lib/stdlib/uc_spec/gen_unicode_mod.escript b/lib/stdlib/uc_spec/gen_unicode_mod.escript index f7c13f0fab..22d21b903b 100644 --- a/lib/stdlib/uc_spec/gen_unicode_mod.escript +++ b/lib/stdlib/uc_spec/gen_unicode_mod.escript @@ -173,7 +173,7 @@ gen_header(Fd) -> io:put_chars(Fd, "-export([whitespace/0, is_whitespace/1]).\n"), io:put_chars(Fd, "-export([uppercase/1, lowercase/1, titlecase/1, casefold/1]).\n\n"), io:put_chars(Fd, "-export([spec_version/0, lookup/1, get_case/1]).\n"), - io:put_chars(Fd, "-inline([class/1]).\n"), + io:put_chars(Fd, "-compile({inline, [class/1]}).\n"), io:put_chars(Fd, "-compile(nowarn_unused_vars).\n"), io:put_chars(Fd, "-dialyzer({no_improper_lists, [cp/1, gc/1, gc_prepend/2]}).\n"), io:put_chars(Fd, "-type gc() :: char()|[char()].\n\n\n"), diff --git a/lib/tools/emacs/erlang.el b/lib/tools/emacs/erlang.el index 69a5fdaeb5..40fe32689e 100644 --- a/lib/tools/emacs/erlang.el +++ b/lib/tools/emacs/erlang.el @@ -5120,8 +5120,8 @@ about Erlang modules." ;; ;; As mentioned this xref implementation is based on the etags xref ;; implementation. But in the cases where arity is considered the -;; etags information structures (class xref-etags-location) will be -;; translated to our own structures which include arity (class +;; etags information structures (struct xref-etags-location) will be +;; translated to our own structures which include arity (struct ;; erlang-xref-location). This translation is started in the function ;; `erlang-refine-xrefs'. @@ -5129,6 +5129,11 @@ about Erlang modules." ;; with xref items with xref-etags-location and some deal with xref ;; items with erlang-xref-location. +;; NOTE: Around Sept 2021, the xrefs package changed all of its defined types +;; (i.e. xref-location, xref-file-location) from EIEIO classes to CL-Lib +;; structures. These are both supported. Older Emacsen with earlier versions of +;; xref will continue to use defclass. Newer Emacsen will use cl-defstruct. + (defun erlang-etags--xref-backend () 'erlang-etags) (defun erlang-soft-require (feature) @@ -5137,6 +5142,7 @@ about Erlang modules." (when (and (erlang-soft-require 'xref) (erlang-soft-require 'cl-generic) + (erlang-soft-require 'cl-lib) (erlang-soft-require 'eieio) (erlang-soft-require 'etags)) ;; The purpose of using eval here is to avoid compilation @@ -5165,10 +5171,20 @@ about Erlang modules." (let ((erlang-replace-etags-tags-completion-table t)) (tags-completion-table))) - (defclass erlang-xref-location (xref-file-location) - ((arity :type fixnum :initarg :arity - :reader erlang-xref-location-arity)) - :documentation "An erlang location is a file location plus arity.") + ;; Xref 1.3.1 bundled with Emacs 28+ switched from using EIEIO classes to + ;; using CL-Lib structs. + (if (find-class 'xref-file-location) + (progn + (defclass erlang-xref-location (xref-file-location) + ((arity :type fixnum :initarg :arity + :reader erlang-xref-location-arity)) + :documentation "An erlang location is a file location plus arity.") + ;; Make a constructor with the same name that a CL structure would have. + (defalias 'make-erlang-xref-location 'erlang-xref-location)) + (cl-defstruct (erlang-xref-location + (:include xref-file-location)) + "An erlang location is a file location plus arity." + (arity 0 :type fixnum))) ;; This method definition only calls the superclass which is ;; the default behaviour if it was not defined. It is only @@ -5331,8 +5347,7 @@ is non-nil then TAG is a regexp." xrefs (when (and xrefs (fboundp 'xref-item-location) - (fboundp 'xref-location-group) - (fboundp 'slot-value)) + (fboundp 'xref-location-group)) (let (files) (cl-loop for xref in xrefs for loc = (xref-item-location xref) @@ -5357,7 +5372,8 @@ is non-nil then TAG is a regexp." t)))) (defun erlang-xrefs-in-file (file kind tag is-regexp) - (when (fboundp 'make-instance) + (when (and (fboundp 'make-erlang-xref-location) + (fboundp 'xref-make)) (with-current-buffer (find-file-noselect file) (save-excursion (goto-char (point-min)) @@ -5369,17 +5385,15 @@ is non-nil then TAG is a regexp." for name = (match-string-no-properties 1) for arity = (save-excursion (erlang-get-arity)) - for loc = (make-instance 'erlang-xref-location - :file file - :line (line-number-at-pos) - :column 0 - :arity arity) + for loc = (make-erlang-xref-location + :file file + :line (line-number-at-pos) + :column 0 + :arity arity) for sum = (erlang-xref-summary kind name arity) when (and arity (not (eq arity last-arity))) - collect (make-instance 'xref-item - :summary sum - :location loc) + collect (xref-make sum loc) do (setq last-arity arity))))))) (defun erlang-xref-summary (kind tag arity) diff --git a/lib/tools/src/instrument.erl b/lib/tools/src/instrument.erl index 1d1edcbbea..31a9925e40 100644 --- a/lib/tools/src/instrument.erl +++ b/lib/tools/src/instrument.erl @@ -46,7 +46,7 @@ allocations() -> allocations(Options) -> Ref = make_ref(), - Defaults = #{ scheduler_ids => lists:seq(0, erlang:system_info(schedulers)), + Defaults = #{ scheduler_ids => lists:seq(0, erts_internal:no_aux_work_threads()-1), allocator_types => erlang:system_info(alloc_util_allocators), histogram_start => 128, histogram_width => 18 }, @@ -118,7 +118,7 @@ carriers() -> carriers(Options) -> Ref = make_ref(), - Defaults = #{ scheduler_ids => lists:seq(0, erlang:system_info(schedulers)), + Defaults = #{ scheduler_ids => lists:seq(0, erts_internal:no_aux_work_threads()-1), allocator_types => erlang:system_info(alloc_util_allocators), histogram_start => 512, histogram_width => 14 }, diff --git a/lib/wx/.dir-locals.el b/lib/wx/.dir-locals.el new file mode 100644 index 0000000000..37133c709f --- /dev/null +++ b/lib/wx/.dir-locals.el @@ -0,0 +1,6 @@ +;; wx Emacs settings +( + ;; In C++ code indentation is 2 spaces + (c++-mode + (indent-tabs-mode . nil) + (c-basic-offset . 2))) diff --git a/lib/wx/.editorconfig b/lib/wx/.editorconfig new file mode 100644 index 0000000000..9729751b71 --- /dev/null +++ b/lib/wx/.editorconfig @@ -0,0 +1,2 @@ +[*.{cpp,hpp}] +indent_size = 2 diff --git a/lib/wx/api_gen/gl_gen_doc.erl b/lib/wx/api_gen/gl_gen_doc.erl index 4307017ff7..83dc2e7236 100644 --- a/lib/wx/api_gen/gl_gen_doc.erl +++ b/lib/wx/api_gen/gl_gen_doc.erl @@ -88,7 +88,7 @@ gen_gl(Name, GLDefs) -> end, ErlRef = {erlref, - [nl(0), {header, wx_gen_doc:gen_header(Name)}, + [nl(0), {header, gen_header(Name)}, nl(0), {module, [Name]}, nl(0), {modulesummary, ["Erlang wrapper functions for OpenGL"]}, nl(0), {description, Desc}, @@ -101,6 +101,28 @@ gen_gl(Name, GLDefs) -> close(), ok. +gen_header(Name) -> + Legal =" + Licensed under the Apache License, Version 2.0 (the \"License\"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an \"AS IS\" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. +", + + [nl(2), {copyright, + [nl(4), {year, ["2020"]}, + nl(4), {holder, ["Ericsson AB. All Rights Reserved."]}]}, + nl(2), {legalnotice, [Legal, nl(2)]}, + nl(2), {title, [Name]}, + nl(0)]. + merge_funcs(All) -> Get = fun(Name0) -> Name = case Name0 of @@ -119,14 +141,17 @@ gen_func({DocName, Fs}) -> put(current_func,DocName), Names0 = [{name, xml_func_name(F#func.name, args(F), 1), []} || F <- Fs], Names = [nl(4)|lists:join(nl(4),Names0)], - [{fsummary,_}=Fsum|Desc] = gen_doc(DocName), - check_doc([Fsum]), - check_doc(Desc), - erase(current_func), - All = [Names, - [nl(4), Fsum, nl(4)], - [{desc, Desc}, nl(2)]], - [nl(2), {func, lists:append(All)}]. + case gen_doc(DocName, Names0) of + ignore -> []; + [{fsummary,_}=Fsum|Desc] -> + check_doc([Fsum]), + check_doc(Desc), + erase(current_func), + All = [Names, + [nl(4), Fsum, nl(4)], + [{desc, Desc}, nl(2)]], + [nl(2), {func, lists:append(All)}] + end. check_doc(Docs) -> try check_doc_1(Docs) @@ -229,23 +254,20 @@ format_func(Func) -> Out(DocExport, Export) end. - - - -gen_doc(Name) -> +gen_doc(Name, _Debug) -> case parse_doc(Name, Dir1 ="gl4", Dir2="gl2.1") of {error, _} when Name =:= "gluTesselate" -> tesselate_doc(); {error, _} -> case reverse(Name) of - "BRA" ++ _ -> ok; - "TXE" ++ _ -> ok; + "BRA" ++ _ -> ignore; + "TXE" ++ _ -> ignore; + "RHK" ++ _ -> ignore; _ -> - %% io:format("Missing doc: no ~s.xml (~s) found in ~s or ~s~n", - %% [Name, Name0, Dir1, Dir2]), - ok - end, - [{fsummary, []}, {p, ["No documentation available."]}]; + %% io:format("Missing doc: ~s not found in ~s or ~s~n ~0.p~n", + %% [Name, Dir1, Dir2, _Debug]), + [{fsummary, []}, {p, ["No documentation available."]}] + end; {Found, Doc} -> {Dir,Ext} = case Found of Dir1 -> {"gl4/html", "xhtml"}; diff --git a/lib/wx/api_gen/gl_gen_erl.erl b/lib/wx/api_gen/gl_gen_erl.erl index 329eca9bd6..950cd85888 100644 --- a/lib/wx/api_gen/gl_gen_erl.erl +++ b/lib/wx/api_gen/gl_gen_erl.erl @@ -37,6 +37,7 @@ -define(HTTP_TOP, "https://www.khronos.org/registry/OpenGL-Refpages/"). +-define(IS_INT(N), N > 47, N < 58). gl_defines(Defs) -> open_write("../include/gl.hrl"), @@ -229,10 +230,9 @@ gen_export2(#func{name=Name,params=As0, alt=Alt}) -> (get({doc_ref,DocN}) == undefined) andalso put({doc_ref, DocN}, Export), Export. -gen_spec([#func{name=Name, params=Orig, alt={vector,VecPos,Vec}}]) -> +gen_spec([#func{name=Name, alt={vector,VecPos,Vec}}]) -> #func{type=T,params=As} = get(Vec), {As1,As2} = lists:split(VecPos, As), - #arg{name=OrigName} = lists:last(Orig), SA1 = case spec_arg_types(As1) of [] -> []; E -> E++", " end, SA2 = spec_arg_types(As2), w("-spec ~s(~s{~s}) -> ~s.~n", [erl_func_name(Name), SA1, SA2, spec_return_types(T,As)]); @@ -438,26 +438,72 @@ check_name("end") -> "'end'"; check_name(Other) -> Other. doc_name(N="glGetBufferParameteriv", _) -> N; +doc_name("glEnableVertexArrayAttrib", _) -> "glEnableVertexAttribArray"; +doc_name("glDisableVertexArrayAttrib", _) -> "glEnableVertexAttribArray"; +doc_name("glEndList",_) -> "glNewList"; doc_name("glEnd"++What, _) -> "glBegin"++What; +doc_name("glDisablei", _) -> "glEnable"; doc_name("glDisable" ++ What, _) -> "glEnable" ++ What; -doc_name("glPop" ++ What, _) -> "glPush" ++ What; -doc_name("glGetBooleanv", _) -> "glGet"; -doc_name("glGetBooleani_v", _) -> "glGet"; -doc_name("glGetIntegerv", _) -> "glGet"; -doc_name("glGetIntegeri_v", _) -> "glGet"; -doc_name("glGetInteger64v", _) -> "glGet"; -doc_name("glGetInteger64i_v", _) -> "glGet"; -doc_name("glGetFloatv", _) -> "glGet"; -doc_name("glGetDoublev", _) -> "glGet"; -doc_name("glGetFloati_v", _) -> "glGet"; -doc_name("glGetDoublei_v", _) -> "glGet"; -doc_name("glUniformMatr" ++ _, _) -> "glUniform"; -doc_name("glTexSubImage" ++ _, _) -> "glTexSubImage"; +doc_name("glClearBuffer" ++ _, _) -> "glClearBuffer"; +doc_name("glCompressedTextureSubImage" ++ End, _) -> "glCompressedTexSubImage" ++ End; doc_name("glFramebufferText" ++ _, _) -> "glFramebufferTexture"; +doc_name("glFramebufferParameteri" ++ _, _) -> "glFramebufferParameteri"; +doc_name("glFlushMappedNamedBufferRange" ++ _, _) -> "glFlushMappedBufferRange"; +doc_name(N="glGetActiveUniformsiv", _) -> N; +doc_name("glGetBoolean" ++ _, _) -> "glGet"; +doc_name("glGetInteger" ++ _, _) -> "glGet"; +doc_name("glGetFloat" ++ _, _) -> "glGet"; +doc_name("glGetDouble" ++ _, _) -> "glGet"; +doc_name("glGetBufferParameter" ++ _, _) -> "glGetBufferParameter"; +doc_name("glGetInternalformat" ++ _, _) -> "glGetInternalFormat"; +doc_name("glGetQueryObject" ++ _, _) -> "glGetQueryObject"; +doc_name("glGetQueryBuffer" ++ _, _) -> "glGetQueryObject"; +doc_name("glGetQueryIndexediv" ++ _, _) -> "glGetQueryIndexed"; +doc_name(N="glGetQuery" ++ _, _) -> N; +doc_name("glGetVertexAttrib" ++ _, _) -> "glGetVertexAttrib"; +doc_name("glGenerateTextureMipmap",_) -> "glGenerateMipmap"; +doc_name("glMap" ++ [I,_], _) -> "glMap" ++ [I]; +doc_name("glMemoryBarrier" ++ _, _) -> "glMemoryBarrier"; +doc_name("glUniformMatr" ++ _, _) -> "glUniform"; +%%doc_name("glTexSubImage" ++ [I,, _) -> "glTexSubImage"; +doc_name("glPop" ++ What, _) -> "glPush" ++ What; doc_name("glProgramUniformMatr" ++ _, _) -> "glProgramUniform"; -doc_name(Name, {has_vector,_,_}) -> +doc_name("glVertexArrayVertexBuffer" ++ S, _) -> "glBindVertexBuffer" ++ S; +doc_name(N="glVertexArrayElementBuffer", _) -> N; +doc_name("glVertexArray" ++ Rest, Alt) -> doc_name("glVertex"++Rest, Alt); +doc_name("glVertexAttrib" ++ [_|"Format"], _) -> "glVertexAttribFormat"; +doc_name("glVertexAttrib" ++ [_|"Pointer"], _) -> "glVertexAttribFormat"; +doc_name("glVertexAttribL" ++ _, _) -> "glVertexAttrib"; +doc_name("glTexture" ++ Rest, _) -> + case Rest of + "Barrier" -> "glTextureBarrier"; + "View" -> "glTextureView"; + _ -> "glTex" ++ Rest + end; +doc_name(Name, Alt) -> + WithoutLast = ["glBlendFunc", "glColorMask", "glClearDepth", + "glCreateShaderProgram", "glDepthRangeArray", + "glEdgeFlag", "glIsEnabled", "glTexEnv", + "glViewportArray", "glScissorArray", "glScissorIndexed" + ], + [Last|RevName] = lists:reverse(Name), + case lists:member(Last, [$i,$f,$v]) of + true -> + New = lists:reverse(RevName), + case lists:member(New, WithoutLast) of + true -> New; + false -> doc_name2(Name,Alt) + end; + false -> + doc_name2(Name,Alt) + end. + +doc_name2(Name, Alt) -> + doc_name_last(Name,Alt). + +doc_name_last(Name, {has_vector,_,_}) -> strip_hard(reverse(Name)); -doc_name(Name, _) -> +doc_name_last(Name, _) -> reverse(strip(reverse(Name))). strip_hard(Rev) -> @@ -467,57 +513,59 @@ strip_hard(Rev) -> end. strip("BRA"++R) -> "BRA"++strip(R); -strip([$v,$b,$u,$N,N|R]) when N > 47, N < 58 ->R; -strip([$v,$i,$u,$N,N|R]) when N > 47, N < 58 ->R; -strip([$v,$s,$u,$N,N|R]) when N > 47, N < 58 ->R; -strip([$v,$b,$N,N|R]) when N > 47, N < 58 -> R; -strip([$v,$i,$N,N|R]) when N > 47, N < 58 -> R; -strip([$v,$s,$N,N|R]) when N > 47, N < 58 -> R; -strip([$v,$d,$N,N|R]) when N > 47, N < 58 -> R; -strip([$v,$f,$N,N|R]) when N > 47, N < 58 -> R; -strip([$b,$u,$N,N|R]) when N > 47, N < 58 -> R; -strip([$v,$b,$I,N|R]) when N > 47, N < 58 -> R; -strip([$v,$i,$I,N|R]) when N > 47, N < 58 -> R; -strip([$v,$s,$I,N|R]) when N > 47, N < 58 -> R; -strip([$v,$d,$I,N|R]) when N > 47, N < 58 -> R; -strip([$v,$f,$I,N|R]) when N > 47, N < 58 -> R; -strip([$b,$u,$I,N|R]) when N > 47, N < 58 -> R; - -strip([$v,$b,$u,N,$I|R]) when N > 47, N < 58 ->R; -strip([$v,$i,$u,N,$I|R]) when N > 47, N < 58 ->R; -strip([$v,$s,$u,N,$I|R]) when N > 47, N < 58 ->R; -strip([$v,$b,N,$I|R]) when N > 47, N < 58 -> R; -strip([$v,$i,N,$I|R]) when N > 47, N < 58 -> R; -strip([$v,$s,N,$I|R]) when N > 47, N < 58 -> R; -strip([$v,$d,N,$I|R]) when N > 47, N < 58 -> R; -strip([$v,$f,N,$I|R]) when N > 47, N < 58 -> R; - -strip([$v,$b,$u,N|R]) when N > 47, N < 58 ->R; -strip([$v,$i,$u,N|R]) when N > 47, N < 58 ->R; -strip([$v,$s,$u,N|R]) when N > 47, N < 58 ->R; -strip([$v,$b,N|R]) when N > 47, N < 58 -> R; -strip([$v,$i,N|R]) when N > 47, N < 58 -> R; -strip([$v,$s,N|R]) when N > 47, N < 58 -> R; -strip([$v,$d,N|R]) when N > 47, N < 58 -> R; -strip([$v,$f,N|R]) when N > 47, N < 58 -> R; - -strip([$b,$u,N,$I|R]) when N > 47, N < 58 ->R; -strip([$i,$u,N,$I|R]) when N > 47, N < 58 ->R; -strip([$s,$u,N,$I|R]) when N > 47, N < 58 ->R; -strip([$b,N,$I|R]) when N > 47, N < 58 -> R; -strip([$i,N,$I|R]) when N > 47, N < 58 -> R; -strip([$s,N,$I|R]) when N > 47, N < 58 -> R; -strip([$d,N,$I|R]) when N > 47, N < 58 -> R; -strip([$f,N,$I|R]) when N > 47, N < 58 -> R; - -strip([$b,$u,N|R]) when N > 47, N < 58 ->R; -strip([$i,$u,N|R]) when N > 47, N < 58 ->R; -strip([$s,$u,N|R]) when N > 47, N < 58 ->R; -strip([$b,N|R]) when N > 47, N < 58 -> R; -strip([$i,N|R]) when N > 47, N < 58 -> R; -strip([$s,N|R]) when N > 47, N < 58 -> R; -strip([$d,N|R]) when N > 47, N < 58 -> R; -strip([$f,N|R]) when N > 47, N < 58 -> R; +strip([$v,$b,$u,$N,N|R]) when ?IS_INT(N) ->R; +strip([$v,$i,$u,$N,N|R]) when ?IS_INT(N) ->R; +strip([$v,$s,$u,$N,N|R]) when ?IS_INT(N) ->R; +strip([$v,$b,$N,N|R]) when ?IS_INT(N) -> R; +strip([$v,$i,$N,N|R]) when ?IS_INT(N) -> R; +strip([$v,$s,$N,N|R]) when ?IS_INT(N) -> R; +strip([$v,$d,$N,N|R]) when ?IS_INT(N) -> R; +strip([$v,$f,$N,N|R]) when ?IS_INT(N) -> R; +strip([$b,$u,$N,N|R]) when ?IS_INT(N) -> R; +strip([$v,$b,$I,N|R]) when ?IS_INT(N) -> R; +strip([$v,$i,$I,N|R]) when ?IS_INT(N) -> R; +strip([$v,$s,$I,N|R]) when ?IS_INT(N) -> R; +strip([$v,$d,$I,N|R]) when ?IS_INT(N) -> R; +strip([$v,$f,$I,N|R]) when ?IS_INT(N) -> R; +strip([$b,$u,$I,N|R]) when ?IS_INT(N) -> R; + +strip([$v,$b,$u,N,$I|R]) when ?IS_INT(N) ->R; +strip([$v,$i,$u,N,$I|R]) when ?IS_INT(N) ->R; +strip([$v,$s,$u,N,$I|R]) when ?IS_INT(N) ->R; +strip([$v,$b,N,$I|R]) when ?IS_INT(N) -> R; +strip([$v,$i,N,$I|R]) when ?IS_INT(N) -> R; +strip([$v,$s,N,$I|R]) when ?IS_INT(N) -> R; +strip([$v,$d,N,$I|R]) when ?IS_INT(N) -> R; +strip([$v,$f,N,$I|R]) when ?IS_INT(N) -> R; + +strip([$v,$b,$u,N|R]) when ?IS_INT(N) ->R; +strip([$v,$i,$u,N|R]) when ?IS_INT(N) ->R; +strip([$v,$s,$u,N|R]) when ?IS_INT(N) ->R; +strip([$v,$b,N|R]) when ?IS_INT(N) -> R; +strip([$v,$i,N|R]) when ?IS_INT(N) -> R; +strip([$v,$s,N|R]) when ?IS_INT(N) -> R; +strip([$v,$d,N|R]) when ?IS_INT(N) -> R; +strip([$v,$f,N|R]) when ?IS_INT(N) -> R; + +strip([$b,$u,N,$I|R]) when ?IS_INT(N) ->R; +strip([$i,$u,N,$I|R]) when ?IS_INT(N) ->R; +strip([$s,$u,N,$I|R]) when ?IS_INT(N) ->R; +strip([$b,N,$I|R]) when ?IS_INT(N) -> R; +strip([$i,N,$I|R]) when ?IS_INT(N) -> R; +strip([$s,N,$I|R]) when ?IS_INT(N) -> R; +strip([$d,N,$I|R]) when ?IS_INT(N) -> R; +strip([$f,N,$I|R]) when ?IS_INT(N) -> R; + +strip([$b,$u,N|R]) when ?IS_INT(N) ->R; +strip([$i,$u,N|R]) when ?IS_INT(N) ->R; +strip([$s,$u,N|R]) when ?IS_INT(N) ->R; +strip([$b,N|R]) when ?IS_INT(N) -> R; +strip([$i,N|R]) when ?IS_INT(N) -> R; +strip([$s,N|R]) when ?IS_INT(N) -> R; +strip([$d,N|R]) when ?IS_INT(N) -> R; +strip([$f,N|R]) when ?IS_INT(N) -> R; + +strip([$v,$i,$u,$I|R]) -> R; strip([$v,$b,$u|R]) -> R; strip([$v,$i,$u|R]) -> R; @@ -529,7 +577,10 @@ strip([$v,$s|R]) -> R; strip([$v,$d|R]) -> R; strip([$v,$f|R]) -> R; -strip(R = "delban" ++ _) -> R; +strip(R = "delban" ++ _) -> R; % E|enabled +strip(R = "dexedn" ++ _) -> R; % I|indexed +strip(R = "decnatsn" ++ _) -> R; % I|instanced + strip([$d,$e|R]) -> [$e|R]; strip([$f,$e|R]) -> [$e|R]; strip([$i,$e|R]) -> [$e|R]; @@ -554,9 +605,9 @@ strip([$d,$n|R]) -> [$n|R]; strip([$I|R]) -> R; strip([$L|R]) -> R; strip([$v,R]) -> R; -strip([N|R]) when N > 47, N < 58 -> R; -strip([_|R="tceRlg"]) -> R; -strip([_|R="thgiLlg"]) -> R; +strip([N|R]) when ?IS_INT(N) -> R; +strip([_|R="tceRlg"]) -> R; % glRect* +strip([_|R="thgiLlg"]) -> R; % glLight* strip(R) -> R. strip2([$b,$u|R]) -> R; diff --git a/lib/wx/api_gen/gl_gen_nif.erl b/lib/wx/api_gen/gl_gen_nif.erl index a7b7b074cc..92d3f71ac9 100644 --- a/lib/wx/api_gen/gl_gen_nif.erl +++ b/lib/wx/api_gen/gl_gen_nif.erl @@ -172,7 +172,7 @@ declare_var(P=#arg{name=Name, false -> w(" ~s ~s;\n",[T,Name]) end, {P,Argc+1}; -declare_var(P=#arg{name=Name,type=#type{base=string,ref={pointer,2},mod=[const]}},Argc) -> +declare_var(P=#arg{name=_Name,type=#type{base=string,ref={pointer,2},mod=[const]}},Argc) -> {P,Argc+1}; declare_var(P=#arg{name=Name, in=true, type=#type{base=Base}}, Argc) when Base =:= binary; Base =:= string -> @@ -181,6 +181,7 @@ declare_var(P=#arg{name=Name, in=true, type=#type{base=Base}}, Argc) declare_var(P=#arg{name=Name, in=true, alt=list_binary, type=#type{name=T}}, Argc) -> w(" ErlNifBinary ~s_bin;\n", [Name]), w(" unsigned int ~s_len;\n", [Name]), + w(" std::vector <~s> ~s_vec;\n", [T, Name]), w(" ~s *~s;\n",[T, Name]), {P,Argc+1}; declare_var(P=#arg{name=Name, in=true, type=#type{name=T, base=guard_int}}, Argc) -> @@ -207,6 +208,10 @@ declare_var(P=#arg{name=Name, in=true, type=#type{name=T,single={Comp, Sz}}}, w(" ~s ~s[~w];\n",[T,Name,Sz]) end, {P,Argc+1}; +declare_var(P=#arg{name=Name, in=true, type=#type{name=Type, single=list}}, Argc) -> + w(" ~s *~s;\n",[Type,Name]), + w(" std::vector <~s> ~s_vec;\n", [Type, Name]), + {P,Argc+1}; declare_var(P=#arg{name=Name, in=true, type=#type{name=T}}, Argc) -> w(" ~s *~s;\n",[T,Name]), {P,Argc+1}; @@ -240,7 +245,7 @@ declare_var(P, Argc) -> %%% Decode -decode_var_0(P=#arg{name=Name}, Argc) -> +decode_var_0(P=#arg{name=_Name}, Argc) -> Res = decode_var(P, Argc), Res. @@ -369,7 +374,6 @@ decode_var(P=#arg{name=Name, in=true, alt=Alt, end, w(" else {\n",[]), w(" ERL_NIF_TERM ~s_l, ~s_h, ~s_t;\n", [Name, Name, Name]), - w(" std::vector <~s> ~s_vec;\n", [Type, Name]), w(" ~s ~s_tmp;\n", [Type, Name]), w(" ~s_l = argv[~w];\n",[Name,Argc]), w(" while(enif_get_list_cell(env, ~s_l, &~s_h, &~s_t)) {\n", [Name,Name,Name]), diff --git a/lib/wx/c_src/gen/gl_nif.cpp b/lib/wx/c_src/gen/gl_nif.cpp index 0db376b754..57e3196d98 100644 --- a/lib/wx/c_src/gen/gl_nif.cpp +++ b/lib/wx/c_src/gen/gl_nif.cpp @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson AB 2008-2020. All Rights Reserved. + * Copyright Ericsson AB 2008-2021. All Rights Reserved. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. @@ -826,11 +826,11 @@ void ecb_glCallLists(ErlNifEnv* env, ErlNifPid *self, ERL_NIF_TERM argv[]) { GLsizei n; GLuint *lists; + std::vector <GLuint> lists_vec; if(!enif_get_int(env, argv[0], &n)) Badarg(5108,"n"); if(!enif_is_list(env, argv[1])) Badarg(5108, "lists") else { ERL_NIF_TERM lists_l, lists_h, lists_t; - std::vector <GLuint> lists_vec; GLuint lists_tmp; lists_l = argv[1]; while(enif_get_list_cell(env, lists_l, &lists_h, &lists_t)) { @@ -2890,11 +2890,11 @@ void ecb_glDeleteTextures(ErlNifEnv* env, ErlNifPid *self, ERL_NIF_TERM argv[]) { GLsizei n; GLuint *textures; + std::vector <GLuint> textures_vec; if(!enif_get_int(env, argv[0], &n)) Badarg(5272,"n"); if(!enif_is_list(env, argv[1])) Badarg(5272, "textures") else { ERL_NIF_TERM textures_l, textures_h, textures_t; - std::vector <GLuint> textures_vec; GLuint textures_tmp; textures_l = argv[1]; while(enif_get_list_cell(env, textures_l, &textures_h, &textures_t)) { @@ -2920,12 +2920,13 @@ void ecb_glPrioritizeTextures(ErlNifEnv* env, ErlNifPid *self, ERL_NIF_TERM argv { GLsizei n; GLuint *textures; + std::vector <GLuint> textures_vec; GLclampf *priorities; + std::vector <GLclampf> priorities_vec; if(!enif_get_int(env, argv[0], &n)) Badarg(5274,"n"); if(!enif_is_list(env, argv[1])) Badarg(5274, "textures") else { ERL_NIF_TERM textures_l, textures_h, textures_t; - std::vector <GLuint> textures_vec; GLuint textures_tmp; textures_l = argv[1]; while(enif_get_list_cell(env, textures_l, &textures_h, &textures_t)) { @@ -2938,7 +2939,6 @@ void ecb_glPrioritizeTextures(ErlNifEnv* env, ErlNifPid *self, ERL_NIF_TERM argv if(!enif_is_list(env, argv[2])) Badarg(5274, "priorities") else { ERL_NIF_TERM priorities_l, priorities_h, priorities_t; - std::vector <GLclampf> priorities_vec; GLclampf priorities_tmp; priorities_l = argv[2]; while(enif_get_list_cell(env, priorities_l, &priorities_h, &priorities_t)) { @@ -2957,11 +2957,11 @@ void ecb_glAreTexturesResident(ErlNifEnv* env, ErlNifPid *self, ERL_NIF_TERM arg ERL_NIF_TERM reply; GLsizei n; GLuint *textures; + std::vector <GLuint> textures_vec; if(!enif_get_int(env, argv[0], &n)) Badarg(5275,"n"); if(!enif_is_list(env, argv[1])) Badarg(5275, "textures") else { ERL_NIF_TERM textures_l, textures_h, textures_t; - std::vector <GLuint> textures_vec; GLuint textures_tmp; textures_l = argv[1]; while(enif_get_list_cell(env, textures_l, &textures_h, &textures_t)) { @@ -4214,9 +4214,11 @@ void ecb_glMultiDrawArrays(ErlNifEnv* env, ErlNifPid *self, ERL_NIF_TERM argv[]) GLenum mode; ErlNifBinary first_bin; unsigned int first_len; + std::vector <GLint> first_vec; GLint *first; ErlNifBinary count_bin; unsigned int count_len; + std::vector <GLsizei> count_vec; GLsizei *count; GLsizei drawcount; if(!enif_get_uint(env, argv[0], &mode)) Badarg(5359,"mode"); @@ -4235,7 +4237,6 @@ void ecb_glMultiDrawArrays(ErlNifEnv* env, ErlNifPid *self, ERL_NIF_TERM argv[]) first_len = first_bin.size / sizeof(GLint); } else { ERL_NIF_TERM first_l, first_h, first_t; - std::vector <GLint> first_vec; GLint first_tmp; first_l = argv[1]; while(enif_get_list_cell(env, first_l, &first_h, &first_t)) { @@ -4261,7 +4262,6 @@ void ecb_glMultiDrawArrays(ErlNifEnv* env, ErlNifPid *self, ERL_NIF_TERM argv[]) count_len = count_bin.size / sizeof(GLsizei); } else { ERL_NIF_TERM count_l, count_h, count_t; - std::vector <GLsizei> count_vec; GLsizei count_tmp; count_l = argv[2]; while(enif_get_list_cell(env, count_l, &count_h, &count_t)) { @@ -4586,11 +4586,11 @@ void ecb_glDeleteQueries(ErlNifEnv* env, ErlNifPid *self, ERL_NIF_TERM argv[]) { GLsizei n; GLuint *ids; + std::vector <GLuint> ids_vec; if(!enif_get_int(env, argv[0], &n)) Badarg(5390,"n"); if(!enif_is_list(env, argv[1])) Badarg(5390, "ids") else { ERL_NIF_TERM ids_l, ids_h, ids_t; - std::vector <GLuint> ids_vec; GLuint ids_tmp; ids_l = argv[1]; while(enif_get_list_cell(env, ids_l, &ids_h, &ids_t)) { @@ -4686,11 +4686,11 @@ void ecb_glDeleteBuffers(ErlNifEnv* env, ErlNifPid *self, ERL_NIF_TERM argv[]) { GLsizei n; GLuint *buffers; + std::vector <GLuint> buffers_vec; if(!enif_get_int(env, argv[0], &n)) Badarg(5398,"n"); if(!enif_is_list(env, argv[1])) Badarg(5398, "buffers") else { ERL_NIF_TERM buffers_l, buffers_h, buffers_t; - std::vector <GLuint> buffers_vec; GLuint buffers_tmp; buffers_l = argv[1]; while(enif_get_list_cell(env, buffers_l, &buffers_h, &buffers_t)) { @@ -4818,11 +4818,11 @@ void ecb_glDrawBuffers(ErlNifEnv* env, ErlNifPid *self, ERL_NIF_TERM argv[]) { GLsizei n; GLenum *bufs; + std::vector <GLenum> bufs_vec; if(!enif_get_int(env, argv[0], &n)) Badarg(5408,"n"); if(!enif_is_list(env, argv[1])) Badarg(5408, "bufs") else { ERL_NIF_TERM bufs_l, bufs_h, bufs_t; - std::vector <GLenum> bufs_vec; GLenum bufs_tmp; bufs_l = argv[1]; while(enif_get_list_cell(env, bufs_l, &bufs_h, &bufs_t)) { @@ -5374,12 +5374,12 @@ void ecb_glUniform1fv(ErlNifEnv* env, ErlNifPid *self, ERL_NIF_TERM argv[]) GLint location; GLsizei count; GLfloat *value; + std::vector <GLfloat> value_vec; if(!enif_get_int(env, argv[0], &location)) Badarg(5450,"location"); if(!enif_get_int(env, argv[1], &count)) Badarg(5450,"count"); if(!enif_is_list(env, argv[2])) Badarg(5450, "value") else { ERL_NIF_TERM value_l, value_h, value_t; - std::vector <GLfloat> value_vec; GLfloat value_tmp; value_l = argv[2]; while(enif_get_list_cell(env, value_l, &value_h, &value_t)) { @@ -5472,12 +5472,12 @@ void ecb_glUniform1iv(ErlNifEnv* env, ErlNifPid *self, ERL_NIF_TERM argv[]) GLint location; GLsizei count; GLint *value; + std::vector <GLint> value_vec; if(!enif_get_int(env, argv[0], &location)) Badarg(5454,"location"); if(!enif_get_int(env, argv[1], &count)) Badarg(5454,"count"); if(!enif_is_list(env, argv[2])) Badarg(5454, "value") else { ERL_NIF_TERM value_l, value_h, value_t; - std::vector <GLint> value_vec; GLint value_tmp; value_l = argv[2]; while(enif_get_list_cell(env, value_l, &value_h, &value_t)) { @@ -6763,12 +6763,12 @@ void ecb_glUniform1uiv(ErlNifEnv* env, ErlNifPid *self, ERL_NIF_TERM argv[]) GLint location; GLsizei count; GLuint *value; + std::vector <GLuint> value_vec; if(!enif_get_int(env, argv[0], &location)) Badarg(5531,"location"); if(!enif_get_int(env, argv[1], &count)) Badarg(5531,"count"); if(!enif_is_list(env, argv[2])) Badarg(5531, "value") else { ERL_NIF_TERM value_l, value_h, value_t; - std::vector <GLuint> value_vec; GLuint value_tmp; value_l = argv[2]; while(enif_get_list_cell(env, value_l, &value_h, &value_t)) { @@ -7044,11 +7044,11 @@ void ecb_glDeleteRenderbuffers(ErlNifEnv* env, ErlNifPid *self, ERL_NIF_TERM arg { GLsizei n; GLuint *renderbuffers; + std::vector <GLuint> renderbuffers_vec; if(!enif_get_int(env, argv[0], &n)) Badarg(5546,"n"); if(!enif_is_list(env, argv[1])) Badarg(5546, "renderbuffers") else { ERL_NIF_TERM renderbuffers_l, renderbuffers_h, renderbuffers_t; - std::vector <GLuint> renderbuffers_vec; GLuint renderbuffers_tmp; renderbuffers_l = argv[1]; while(enif_get_list_cell(env, renderbuffers_l, &renderbuffers_h, &renderbuffers_t)) { @@ -7128,11 +7128,11 @@ void ecb_glDeleteFramebuffers(ErlNifEnv* env, ErlNifPid *self, ERL_NIF_TERM argv { GLsizei n; GLuint *framebuffers; + std::vector <GLuint> framebuffers_vec; if(!enif_get_int(env, argv[0], &n)) Badarg(5552,"n"); if(!enif_is_list(env, argv[1])) Badarg(5552, "framebuffers") else { ERL_NIF_TERM framebuffers_l, framebuffers_h, framebuffers_t; - std::vector <GLuint> framebuffers_vec; GLuint framebuffers_tmp; framebuffers_l = argv[1]; while(enif_get_list_cell(env, framebuffers_l, &framebuffers_h, &framebuffers_t)) { @@ -7332,11 +7332,11 @@ void ecb_glDeleteVertexArrays(ErlNifEnv* env, ErlNifPid *self, ERL_NIF_TERM argv { GLsizei n; GLuint *arrays; + std::vector <GLuint> arrays_vec; if(!enif_get_int(env, argv[0], &n)) Badarg(5566,"n"); if(!enif_is_list(env, argv[1])) Badarg(5566, "arrays") else { ERL_NIF_TERM arrays_l, arrays_h, arrays_t; - std::vector <GLuint> arrays_vec; GLuint arrays_tmp; arrays_l = argv[1]; while(enif_get_list_cell(env, arrays_l, &arrays_h, &arrays_t)) { @@ -7474,13 +7474,13 @@ void ecb_glGetActiveUniformsiv(ErlNifEnv* env, ErlNifPid *self, ERL_NIF_TERM arg GLuint program; GLsizei uniformCount; GLuint *uniformIndices; + std::vector <GLuint> uniformIndices_vec; GLenum pname; if(!enif_get_uint(env, argv[0], &program)) Badarg(5576,"program"); if(!enif_get_int(env, argv[1], &uniformCount)) Badarg(5576,"uniformCount"); if(!enif_is_list(env, argv[2])) Badarg(5576, "uniformIndices") else { ERL_NIF_TERM uniformIndices_l, uniformIndices_h, uniformIndices_t; - std::vector <GLuint> uniformIndices_vec; GLuint uniformIndices_tmp; uniformIndices_l = argv[2]; while(enif_get_list_cell(env, uniformIndices_l, &uniformIndices_h, &uniformIndices_t)) { @@ -7911,11 +7911,11 @@ void ecb_glDeleteSamplers(ErlNifEnv* env, ErlNifPid *self, ERL_NIF_TERM argv[]) { GLsizei count; GLuint *samplers; + std::vector <GLuint> samplers_vec; if(!enif_get_int(env, argv[0], &count)) Badarg(5606,"count"); if(!enif_is_list(env, argv[1])) Badarg(5606, "samplers") else { ERL_NIF_TERM samplers_l, samplers_h, samplers_t; - std::vector <GLuint> samplers_vec; GLuint samplers_tmp; samplers_l = argv[1]; while(enif_get_list_cell(env, samplers_l, &samplers_h, &samplers_t)) { @@ -7965,12 +7965,12 @@ void ecb_glSamplerParameteriv(ErlNifEnv* env, ErlNifPid *self, ERL_NIF_TERM argv GLuint sampler; GLenum pname; GLint *param; + std::vector <GLint> param_vec; if(!enif_get_uint(env, argv[0], &sampler)) Badarg(5610,"sampler"); if(!enif_get_uint(env, argv[1], &pname)) Badarg(5610,"pname"); if(!enif_is_list(env, argv[2])) Badarg(5610, "param") else { ERL_NIF_TERM param_l, param_h, param_t; - std::vector <GLint> param_vec; GLint param_tmp; param_l = argv[2]; while(enif_get_list_cell(env, param_l, ¶m_h, ¶m_t)) { @@ -7999,12 +7999,12 @@ void ecb_glSamplerParameterfv(ErlNifEnv* env, ErlNifPid *self, ERL_NIF_TERM argv GLuint sampler; GLenum pname; GLfloat *param; + std::vector <GLfloat> param_vec; if(!enif_get_uint(env, argv[0], &sampler)) Badarg(5612,"sampler"); if(!enif_get_uint(env, argv[1], &pname)) Badarg(5612,"pname"); if(!enif_is_list(env, argv[2])) Badarg(5612, "param") else { ERL_NIF_TERM param_l, param_h, param_t; - std::vector <GLfloat> param_vec; GLfloat param_tmp; param_l = argv[2]; while(enif_get_list_cell(env, param_l, ¶m_h, ¶m_t)) { @@ -8022,12 +8022,12 @@ void ecb_glSamplerParameterIiv(ErlNifEnv* env, ErlNifPid *self, ERL_NIF_TERM arg GLuint sampler; GLenum pname; GLint *param; + std::vector <GLint> param_vec; if(!enif_get_uint(env, argv[0], &sampler)) Badarg(5613,"sampler"); if(!enif_get_uint(env, argv[1], &pname)) Badarg(5613,"pname"); if(!enif_is_list(env, argv[2])) Badarg(5613, "param") else { ERL_NIF_TERM param_l, param_h, param_t; - std::vector <GLint> param_vec; GLint param_tmp; param_l = argv[2]; while(enif_get_list_cell(env, param_l, ¶m_h, ¶m_t)) { @@ -8045,12 +8045,12 @@ void ecb_glSamplerParameterIuiv(ErlNifEnv* env, ErlNifPid *self, ERL_NIF_TERM ar GLuint sampler; GLenum pname; GLuint *param; + std::vector <GLuint> param_vec; if(!enif_get_uint(env, argv[0], &sampler)) Badarg(5614,"sampler"); if(!enif_get_uint(env, argv[1], &pname)) Badarg(5614,"pname"); if(!enif_is_list(env, argv[2])) Badarg(5614, "param") else { ERL_NIF_TERM param_l, param_h, param_t; - std::vector <GLuint> param_vec; GLuint param_tmp; param_l = argv[2]; while(enif_get_list_cell(env, param_l, ¶m_h, ¶m_t)) { @@ -8317,12 +8317,12 @@ void ecb_glUniform1dv(ErlNifEnv* env, ErlNifPid *self, ERL_NIF_TERM argv[]) GLint location; GLsizei count; GLdouble *value; + std::vector <GLdouble> value_vec; if(!enif_get_int(env, argv[0], &location)) Badarg(5636,"location"); if(!enif_get_int(env, argv[1], &count)) Badarg(5636,"count"); if(!enif_is_list(env, argv[2])) Badarg(5636, "value") else { ERL_NIF_TERM value_l, value_h, value_t; - std::vector <GLdouble> value_vec; GLdouble value_tmp; value_l = argv[2]; while(enif_get_list_cell(env, value_l, &value_h, &value_t)) { @@ -8803,12 +8803,12 @@ void ecb_glUniformSubroutinesuiv(ErlNifEnv* env, ErlNifPid *self, ERL_NIF_TERM a GLenum shadertype; GLsizei count; GLuint *indices; + std::vector <GLuint> indices_vec; if(!enif_get_uint(env, argv[0], &shadertype)) Badarg(5654,"shadertype"); if(!enif_get_int(env, argv[1], &count)) Badarg(5654,"count"); if(!enif_is_list(env, argv[2])) Badarg(5654, "indices") else { ERL_NIF_TERM indices_l, indices_h, indices_t; - std::vector <GLuint> indices_vec; GLuint indices_tmp; indices_l = argv[2]; while(enif_get_list_cell(env, indices_l, &indices_h, &indices_t)) { @@ -8867,11 +8867,11 @@ void ecb_glPatchParameterfv(ErlNifEnv* env, ErlNifPid *self, ERL_NIF_TERM argv[] { GLenum pname; GLfloat *values; + std::vector <GLfloat> values_vec; if(!enif_get_uint(env, argv[0], &pname)) Badarg(5658,"pname"); if(!enif_is_list(env, argv[1])) Badarg(5658, "values") else { ERL_NIF_TERM values_l, values_h, values_t; - std::vector <GLfloat> values_vec; GLfloat values_tmp; values_l = argv[1]; while(enif_get_list_cell(env, values_l, &values_h, &values_t)) { @@ -8897,11 +8897,11 @@ void ecb_glDeleteTransformFeedbacks(ErlNifEnv* env, ErlNifPid *self, ERL_NIF_TER { GLsizei n; GLuint *ids; + std::vector <GLuint> ids_vec; if(!enif_get_int(env, argv[0], &n)) Badarg(5660,"n"); if(!enif_is_list(env, argv[1])) Badarg(5660, "ids") else { ERL_NIF_TERM ids_l, ids_h, ids_t; - std::vector <GLuint> ids_vec; GLuint ids_tmp; ids_l = argv[1]; while(enif_get_list_cell(env, ids_l, &ids_h, &ids_t)) { @@ -9016,13 +9016,13 @@ void ecb_glShaderBinary(ErlNifEnv* env, ErlNifPid *self, ERL_NIF_TERM argv[]) { GLsizei count; GLuint *shaders; + std::vector <GLuint> shaders_vec; GLenum binaryformat; ErlNifBinary binary; if(!enif_get_int(env, argv[0], &count)) Badarg(5671,"count"); if(!enif_is_list(env, argv[1])) Badarg(5671, "shaders") else { ERL_NIF_TERM shaders_l, shaders_h, shaders_t; - std::vector <GLuint> shaders_vec; GLuint shaders_tmp; shaders_l = argv[1]; while(enif_get_list_cell(env, shaders_l, &shaders_h, &shaders_t)) { @@ -9167,11 +9167,11 @@ void ecb_glDeleteProgramPipelines(ErlNifEnv* env, ErlNifPid *self, ERL_NIF_TERM { GLsizei n; GLuint *pipelines; + std::vector <GLuint> pipelines_vec; if(!enif_get_int(env, argv[0], &n)) Badarg(5682,"n"); if(!enif_is_list(env, argv[1])) Badarg(5682, "pipelines") else { ERL_NIF_TERM pipelines_l, pipelines_h, pipelines_t; - std::vector <GLuint> pipelines_vec; GLuint pipelines_tmp; pipelines_l = argv[1]; while(enif_get_list_cell(env, pipelines_l, &pipelines_h, &pipelines_t)) { @@ -9242,13 +9242,13 @@ void ecb_glProgramUniform1iv(ErlNifEnv* env, ErlNifPid *self, ERL_NIF_TERM argv[ GLint location; GLsizei count; GLint *value; + std::vector <GLint> value_vec; if(!enif_get_uint(env, argv[0], &program)) Badarg(5687,"program"); if(!enif_get_int(env, argv[1], &location)) Badarg(5687,"location"); if(!enif_get_int(env, argv[2], &count)) Badarg(5687,"count"); if(!enif_is_list(env, argv[3])) Badarg(5687, "value") else { ERL_NIF_TERM value_l, value_h, value_t; - std::vector <GLint> value_vec; GLint value_tmp; value_l = argv[3]; while(enif_get_list_cell(env, value_l, &value_h, &value_t)) { @@ -9278,13 +9278,13 @@ void ecb_glProgramUniform1fv(ErlNifEnv* env, ErlNifPid *self, ERL_NIF_TERM argv[ GLint location; GLsizei count; GLfloat *value; + std::vector <GLfloat> value_vec; if(!enif_get_uint(env, argv[0], &program)) Badarg(5689,"program"); if(!enif_get_int(env, argv[1], &location)) Badarg(5689,"location"); if(!enif_get_int(env, argv[2], &count)) Badarg(5689,"count"); if(!enif_is_list(env, argv[3])) Badarg(5689, "value") else { ERL_NIF_TERM value_l, value_h, value_t; - std::vector <GLfloat> value_vec; GLfloat value_tmp; value_l = argv[3]; while(enif_get_list_cell(env, value_l, &value_h, &value_t)) { @@ -9314,13 +9314,13 @@ void ecb_glProgramUniform1dv(ErlNifEnv* env, ErlNifPid *self, ERL_NIF_TERM argv[ GLint location; GLsizei count; GLdouble *value; + std::vector <GLdouble> value_vec; if(!enif_get_uint(env, argv[0], &program)) Badarg(5691,"program"); if(!enif_get_int(env, argv[1], &location)) Badarg(5691,"location"); if(!enif_get_int(env, argv[2], &count)) Badarg(5691,"count"); if(!enif_is_list(env, argv[3])) Badarg(5691, "value") else { ERL_NIF_TERM value_l, value_h, value_t; - std::vector <GLdouble> value_vec; GLdouble value_tmp; value_l = argv[3]; while(enif_get_list_cell(env, value_l, &value_h, &value_t)) { @@ -9350,13 +9350,13 @@ void ecb_glProgramUniform1uiv(ErlNifEnv* env, ErlNifPid *self, ERL_NIF_TERM argv GLint location; GLsizei count; GLuint *value; + std::vector <GLuint> value_vec; if(!enif_get_uint(env, argv[0], &program)) Badarg(5693,"program"); if(!enif_get_int(env, argv[1], &location)) Badarg(5693,"location"); if(!enif_get_int(env, argv[2], &count)) Badarg(5693,"count"); if(!enif_is_list(env, argv[3])) Badarg(5693, "value") else { ERL_NIF_TERM value_l, value_h, value_t; - std::vector <GLuint> value_vec; GLuint value_tmp; value_l = argv[3]; while(enif_get_list_cell(env, value_l, &value_h, &value_t)) { @@ -11177,12 +11177,12 @@ void ecb_glInvalidateFramebuffer(ErlNifEnv* env, ErlNifPid *self, ERL_NIF_TERM a GLenum target; GLsizei numAttachments; GLenum *attachments; + std::vector <GLenum> attachments_vec; if(!enif_get_uint(env, argv[0], &target)) Badarg(5782,"target"); if(!enif_get_int(env, argv[1], &numAttachments)) Badarg(5782,"numAttachments"); if(!enif_is_list(env, argv[2])) Badarg(5782, "attachments") else { ERL_NIF_TERM attachments_l, attachments_h, attachments_t; - std::vector <GLenum> attachments_vec; GLenum attachments_tmp; attachments_l = argv[2]; while(enif_get_list_cell(env, attachments_l, &attachments_h, &attachments_t)) { @@ -11200,6 +11200,7 @@ void ecb_glInvalidateSubFramebuffer(ErlNifEnv* env, ErlNifPid *self, ERL_NIF_TER GLenum target; GLsizei numAttachments; GLenum *attachments; + std::vector <GLenum> attachments_vec; GLint x; GLint y; GLsizei width; @@ -11209,7 +11210,6 @@ void ecb_glInvalidateSubFramebuffer(ErlNifEnv* env, ErlNifPid *self, ERL_NIF_TER if(!enif_is_list(env, argv[2])) Badarg(5783, "attachments") else { ERL_NIF_TERM attachments_l, attachments_h, attachments_t; - std::vector <GLenum> attachments_vec; GLenum attachments_tmp; attachments_l = argv[2]; while(enif_get_list_cell(env, attachments_l, &attachments_h, &attachments_t)) { @@ -11490,6 +11490,7 @@ void ecb_glDebugMessageControl(ErlNifEnv* env, ErlNifPid *self, ERL_NIF_TERM arg GLenum severity; GLsizei count; GLuint *ids; + std::vector <GLuint> ids_vec; GLboolean enabled; if(!enif_get_uint(env, argv[0], &source)) Badarg(5802,"source"); if(!enif_get_uint(env, argv[1], &type)) Badarg(5802,"type"); @@ -11498,7 +11499,6 @@ void ecb_glDebugMessageControl(ErlNifEnv* env, ErlNifPid *self, ERL_NIF_TERM arg if(!enif_is_list(env, argv[4])) Badarg(5802, "ids") else { ERL_NIF_TERM ids_l, ids_h, ids_t; - std::vector <GLuint> ids_vec; GLuint ids_tmp; ids_l = argv[4]; while(enif_get_list_cell(env, ids_l, &ids_h, &ids_t)) { @@ -11680,13 +11680,13 @@ void ecb_glBindBuffersBase(ErlNifEnv* env, ErlNifPid *self, ERL_NIF_TERM argv[]) GLuint first; GLsizei count; GLuint *buffers; + std::vector <GLuint> buffers_vec; if(!enif_get_uint(env, argv[0], &target)) Badarg(5815,"target"); if(!enif_get_uint(env, argv[1], &first)) Badarg(5815,"first"); if(!enif_get_int(env, argv[2], &count)) Badarg(5815,"count"); if(!enif_is_list(env, argv[3])) Badarg(5815, "buffers") else { ERL_NIF_TERM buffers_l, buffers_h, buffers_t; - std::vector <GLuint> buffers_vec; GLuint buffers_tmp; buffers_l = argv[3]; while(enif_get_list_cell(env, buffers_l, &buffers_h, &buffers_t)) { @@ -11705,15 +11705,17 @@ void ecb_glBindBuffersRange(ErlNifEnv* env, ErlNifPid *self, ERL_NIF_TERM argv[] GLuint first; GLsizei count; GLuint *buffers; + std::vector <GLuint> buffers_vec; GLintptr *offsets; + std::vector <GLintptr> offsets_vec; GLsizeiptr *sizes; + std::vector <GLsizeiptr> sizes_vec; if(!enif_get_uint(env, argv[0], &target)) Badarg(5816,"target"); if(!enif_get_uint(env, argv[1], &first)) Badarg(5816,"first"); if(!enif_get_int(env, argv[2], &count)) Badarg(5816,"count"); if(!enif_is_list(env, argv[3])) Badarg(5816, "buffers") else { ERL_NIF_TERM buffers_l, buffers_h, buffers_t; - std::vector <GLuint> buffers_vec; GLuint buffers_tmp; buffers_l = argv[3]; while(enif_get_list_cell(env, buffers_l, &buffers_h, &buffers_t)) { @@ -11726,7 +11728,6 @@ void ecb_glBindBuffersRange(ErlNifEnv* env, ErlNifPid *self, ERL_NIF_TERM argv[] if(!enif_is_list(env, argv[4])) Badarg(5816, "offsets") else { ERL_NIF_TERM offsets_l, offsets_h, offsets_t; - std::vector <GLintptr> offsets_vec; GLintptr offsets_tmp; offsets_l = argv[4]; while(enif_get_list_cell(env, offsets_l, &offsets_h, &offsets_t)) { @@ -11739,7 +11740,6 @@ void ecb_glBindBuffersRange(ErlNifEnv* env, ErlNifPid *self, ERL_NIF_TERM argv[] if(!enif_is_list(env, argv[5])) Badarg(5816, "sizes") else { ERL_NIF_TERM sizes_l, sizes_h, sizes_t; - std::vector <GLsizeiptr> sizes_vec; GLsizeiptr sizes_tmp; sizes_l = argv[5]; while(enif_get_list_cell(env, sizes_l, &sizes_h, &sizes_t)) { @@ -11757,12 +11757,12 @@ void ecb_glBindTextures(ErlNifEnv* env, ErlNifPid *self, ERL_NIF_TERM argv[]) GLuint first; GLsizei count; GLuint *textures; + std::vector <GLuint> textures_vec; if(!enif_get_uint(env, argv[0], &first)) Badarg(5817,"first"); if(!enif_get_int(env, argv[1], &count)) Badarg(5817,"count"); if(!enif_is_list(env, argv[2])) Badarg(5817, "textures") else { ERL_NIF_TERM textures_l, textures_h, textures_t; - std::vector <GLuint> textures_vec; GLuint textures_tmp; textures_l = argv[2]; while(enif_get_list_cell(env, textures_l, &textures_h, &textures_t)) { @@ -11780,12 +11780,12 @@ void ecb_glBindSamplers(ErlNifEnv* env, ErlNifPid *self, ERL_NIF_TERM argv[]) GLuint first; GLsizei count; GLuint *samplers; + std::vector <GLuint> samplers_vec; if(!enif_get_uint(env, argv[0], &first)) Badarg(5818,"first"); if(!enif_get_int(env, argv[1], &count)) Badarg(5818,"count"); if(!enif_is_list(env, argv[2])) Badarg(5818, "samplers") else { ERL_NIF_TERM samplers_l, samplers_h, samplers_t; - std::vector <GLuint> samplers_vec; GLuint samplers_tmp; samplers_l = argv[2]; while(enif_get_list_cell(env, samplers_l, &samplers_h, &samplers_t)) { @@ -11803,12 +11803,12 @@ void ecb_glBindImageTextures(ErlNifEnv* env, ErlNifPid *self, ERL_NIF_TERM argv[ GLuint first; GLsizei count; GLuint *textures; + std::vector <GLuint> textures_vec; if(!enif_get_uint(env, argv[0], &first)) Badarg(5819,"first"); if(!enif_get_int(env, argv[1], &count)) Badarg(5819,"count"); if(!enif_is_list(env, argv[2])) Badarg(5819, "textures") else { ERL_NIF_TERM textures_l, textures_h, textures_t; - std::vector <GLuint> textures_vec; GLuint textures_tmp; textures_l = argv[2]; while(enif_get_list_cell(env, textures_l, &textures_h, &textures_t)) { @@ -11826,14 +11826,16 @@ void ecb_glBindVertexBuffers(ErlNifEnv* env, ErlNifPid *self, ERL_NIF_TERM argv[ GLuint first; GLsizei count; GLuint *buffers; + std::vector <GLuint> buffers_vec; GLintptr *offsets; + std::vector <GLintptr> offsets_vec; GLsizei *strides; + std::vector <GLsizei> strides_vec; if(!enif_get_uint(env, argv[0], &first)) Badarg(5820,"first"); if(!enif_get_int(env, argv[1], &count)) Badarg(5820,"count"); if(!enif_is_list(env, argv[2])) Badarg(5820, "buffers") else { ERL_NIF_TERM buffers_l, buffers_h, buffers_t; - std::vector <GLuint> buffers_vec; GLuint buffers_tmp; buffers_l = argv[2]; while(enif_get_list_cell(env, buffers_l, &buffers_h, &buffers_t)) { @@ -11846,7 +11848,6 @@ void ecb_glBindVertexBuffers(ErlNifEnv* env, ErlNifPid *self, ERL_NIF_TERM argv[ if(!enif_is_list(env, argv[3])) Badarg(5820, "offsets") else { ERL_NIF_TERM offsets_l, offsets_h, offsets_t; - std::vector <GLintptr> offsets_vec; GLintptr offsets_tmp; offsets_l = argv[3]; while(enif_get_list_cell(env, offsets_l, &offsets_h, &offsets_t)) { @@ -11859,7 +11860,6 @@ void ecb_glBindVertexBuffers(ErlNifEnv* env, ErlNifPid *self, ERL_NIF_TERM argv[ if(!enif_is_list(env, argv[4])) Badarg(5820, "strides") else { ERL_NIF_TERM strides_l, strides_h, strides_t; - std::vector <GLsizei> strides_vec; GLsizei strides_tmp; strides_l = argv[4]; while(enif_get_list_cell(env, strides_l, &strides_h, &strides_t)) { @@ -12184,15 +12184,17 @@ void ecb_glVertexArrayVertexBuffers(ErlNifEnv* env, ErlNifPid *self, ERL_NIF_TER GLuint first; GLsizei count; GLuint *buffers; + std::vector <GLuint> buffers_vec; GLintptr *offsets; + std::vector <GLintptr> offsets_vec; GLsizei *strides; + std::vector <GLsizei> strides_vec; if(!enif_get_uint(env, argv[0], &vaobj)) Badarg(5845,"vaobj"); if(!enif_get_uint(env, argv[1], &first)) Badarg(5845,"first"); if(!enif_get_int(env, argv[2], &count)) Badarg(5845,"count"); if(!enif_is_list(env, argv[3])) Badarg(5845, "buffers") else { ERL_NIF_TERM buffers_l, buffers_h, buffers_t; - std::vector <GLuint> buffers_vec; GLuint buffers_tmp; buffers_l = argv[3]; while(enif_get_list_cell(env, buffers_l, &buffers_h, &buffers_t)) { @@ -12205,7 +12207,6 @@ void ecb_glVertexArrayVertexBuffers(ErlNifEnv* env, ErlNifPid *self, ERL_NIF_TER if(!enif_is_list(env, argv[4])) Badarg(5845, "offsets") else { ERL_NIF_TERM offsets_l, offsets_h, offsets_t; - std::vector <GLintptr> offsets_vec; GLintptr offsets_tmp; offsets_l = argv[4]; while(enif_get_list_cell(env, offsets_l, &offsets_h, &offsets_t)) { @@ -12218,7 +12219,6 @@ void ecb_glVertexArrayVertexBuffers(ErlNifEnv* env, ErlNifPid *self, ERL_NIF_TER if(!enif_is_list(env, argv[5])) Badarg(5845, "strides") else { ERL_NIF_TERM strides_l, strides_h, strides_t; - std::vector <GLsizei> strides_vec; GLsizei strides_tmp; strides_l = argv[5]; while(enif_get_list_cell(env, strides_l, &strides_h, &strides_t)) { @@ -12608,11 +12608,11 @@ void ecb_glDeleteProgramsARB(ErlNifEnv* env, ErlNifPid *self, ERL_NIF_TERM argv[ { GLsizei n; GLuint *programs; + std::vector <GLuint> programs_vec; if(!enif_get_int(env, argv[0], &n)) Badarg(5877,"n"); if(!enif_is_list(env, argv[1])) Badarg(5877, "programs") else { ERL_NIF_TERM programs_l, programs_h, programs_t; - std::vector <GLuint> programs_vec; GLuint programs_tmp; programs_l = argv[1]; while(enif_get_list_cell(env, programs_l, &programs_h, &programs_t)) { @@ -12959,12 +12959,12 @@ void ecb_glUniform1i64vARB(ErlNifEnv* env, ErlNifPid *self, ERL_NIF_TERM argv[]) GLint location; GLsizei count; GLint64 *value; + std::vector <GLint64> value_vec; if(!enif_get_int(env, argv[0], &location)) Badarg(5897,"location"); if(!enif_get_int(env, argv[1], &count)) Badarg(5897,"count"); if(!enif_is_list(env, argv[2])) Badarg(5897, "value") else { ERL_NIF_TERM value_l, value_h, value_t; - std::vector <GLint64> value_vec; GLint64 value_tmp; value_l = argv[2]; while(enif_get_list_cell(env, value_l, &value_h, &value_t)) { @@ -13105,12 +13105,12 @@ void ecb_glUniform1ui64vARB(ErlNifEnv* env, ErlNifPid *self, ERL_NIF_TERM argv[] GLint location; GLsizei count; GLuint64 *value; + std::vector <GLuint64> value_vec; if(!enif_get_int(env, argv[0], &location)) Badarg(5905,"location"); if(!enif_get_int(env, argv[1], &count)) Badarg(5905,"count"); if(!enif_is_list(env, argv[2])) Badarg(5905, "value") else { ERL_NIF_TERM value_l, value_h, value_t; - std::vector <GLuint64> value_vec; GLuint64 value_tmp; value_l = argv[2]; while(enif_get_list_cell(env, value_l, &value_h, &value_t)) { @@ -13294,13 +13294,13 @@ void ecb_glProgramUniform1i64vARB(ErlNifEnv* env, ErlNifPid *self, ERL_NIF_TERM GLint location; GLsizei count; GLint64 *value; + std::vector <GLint64> value_vec; if(!enif_get_uint(env, argv[0], &program)) Badarg(5915,"program"); if(!enif_get_int(env, argv[1], &location)) Badarg(5915,"location"); if(!enif_get_int(env, argv[2], &count)) Badarg(5915,"count"); if(!enif_is_list(env, argv[3])) Badarg(5915, "value") else { ERL_NIF_TERM value_l, value_h, value_t; - std::vector <GLint64> value_vec; GLint64 value_tmp; value_l = argv[3]; while(enif_get_list_cell(env, value_l, &value_h, &value_t)) { @@ -13456,13 +13456,13 @@ void ecb_glProgramUniform1ui64vARB(ErlNifEnv* env, ErlNifPid *self, ERL_NIF_TERM GLint location; GLsizei count; GLuint64 *value; + std::vector <GLuint64> value_vec; if(!enif_get_uint(env, argv[0], &program)) Badarg(5923,"program"); if(!enif_get_int(env, argv[1], &location)) Badarg(5923,"location"); if(!enif_get_int(env, argv[2], &count)) Badarg(5923,"count"); if(!enif_is_list(env, argv[3])) Badarg(5923, "value") else { ERL_NIF_TERM value_l, value_h, value_t; - std::vector <GLuint64> value_vec; GLuint64 value_tmp; value_l = argv[3]; while(enif_get_list_cell(env, value_l, &value_h, &value_t)) { @@ -14148,11 +14148,11 @@ void ecb_glMatrixIndexubvARB(ErlNifEnv* env, ErlNifPid *self, ERL_NIF_TERM argv[ { GLint size; GLubyte *indices; + std::vector <GLubyte> indices_vec; if(!enif_get_int(env, argv[0], &size)) Badarg(5964,"size"); if(!enif_is_list(env, argv[1])) Badarg(5964, "indices") else { ERL_NIF_TERM indices_l, indices_h, indices_t; - std::vector <GLubyte> indices_vec; GLubyte indices_tmp; indices_l = argv[1]; while(enif_get_list_cell(env, indices_l, &indices_h, &indices_t)) { @@ -14169,11 +14169,11 @@ void ecb_glMatrixIndexusvARB(ErlNifEnv* env, ErlNifPid *self, ERL_NIF_TERM argv[ { GLint size; GLushort *indices; + std::vector <GLushort> indices_vec; if(!enif_get_int(env, argv[0], &size)) Badarg(5965,"size"); if(!enif_is_list(env, argv[1])) Badarg(5965, "indices") else { ERL_NIF_TERM indices_l, indices_h, indices_t; - std::vector <GLushort> indices_vec; GLushort indices_tmp; indices_l = argv[1]; while(enif_get_list_cell(env, indices_l, &indices_h, &indices_t)) { @@ -14190,11 +14190,11 @@ void ecb_glMatrixIndexuivARB(ErlNifEnv* env, ErlNifPid *self, ERL_NIF_TERM argv[ { GLint size; GLuint *indices; + std::vector <GLuint> indices_vec; if(!enif_get_int(env, argv[0], &size)) Badarg(5966,"size"); if(!enif_is_list(env, argv[1])) Badarg(5966, "indices") else { ERL_NIF_TERM indices_l, indices_h, indices_t; - std::vector <GLuint> indices_vec; GLuint indices_tmp; indices_l = argv[1]; while(enif_get_list_cell(env, indices_l, &indices_h, &indices_t)) { @@ -14737,11 +14737,11 @@ void ecb_glWeightbvARB(ErlNifEnv* env, ErlNifPid *self, ERL_NIF_TERM argv[]) { GLint size; GLbyte *weights; + std::vector <GLbyte> weights_vec; if(!enif_get_int(env, argv[0], &size)) Badarg(6000,"size"); if(!enif_is_list(env, argv[1])) Badarg(6000, "weights") else { ERL_NIF_TERM weights_l, weights_h, weights_t; - std::vector <GLbyte> weights_vec; GLbyte weights_tmp; weights_l = argv[1]; while(enif_get_list_cell(env, weights_l, &weights_h, &weights_t)) { @@ -14758,11 +14758,11 @@ void ecb_glWeightsvARB(ErlNifEnv* env, ErlNifPid *self, ERL_NIF_TERM argv[]) { GLint size; GLshort *weights; + std::vector <GLshort> weights_vec; if(!enif_get_int(env, argv[0], &size)) Badarg(6001,"size"); if(!enif_is_list(env, argv[1])) Badarg(6001, "weights") else { ERL_NIF_TERM weights_l, weights_h, weights_t; - std::vector <GLshort> weights_vec; GLshort weights_tmp; weights_l = argv[1]; while(enif_get_list_cell(env, weights_l, &weights_h, &weights_t)) { @@ -14779,11 +14779,11 @@ void ecb_glWeightivARB(ErlNifEnv* env, ErlNifPid *self, ERL_NIF_TERM argv[]) { GLint size; GLint *weights; + std::vector <GLint> weights_vec; if(!enif_get_int(env, argv[0], &size)) Badarg(6002,"size"); if(!enif_is_list(env, argv[1])) Badarg(6002, "weights") else { ERL_NIF_TERM weights_l, weights_h, weights_t; - std::vector <GLint> weights_vec; GLint weights_tmp; weights_l = argv[1]; while(enif_get_list_cell(env, weights_l, &weights_h, &weights_t)) { @@ -14800,11 +14800,11 @@ void ecb_glWeightfvARB(ErlNifEnv* env, ErlNifPid *self, ERL_NIF_TERM argv[]) { GLint size; GLfloat *weights; + std::vector <GLfloat> weights_vec; if(!enif_get_int(env, argv[0], &size)) Badarg(6003,"size"); if(!enif_is_list(env, argv[1])) Badarg(6003, "weights") else { ERL_NIF_TERM weights_l, weights_h, weights_t; - std::vector <GLfloat> weights_vec; GLfloat weights_tmp; weights_l = argv[1]; while(enif_get_list_cell(env, weights_l, &weights_h, &weights_t)) { @@ -14821,11 +14821,11 @@ void ecb_glWeightdvARB(ErlNifEnv* env, ErlNifPid *self, ERL_NIF_TERM argv[]) { GLint size; GLdouble *weights; + std::vector <GLdouble> weights_vec; if(!enif_get_int(env, argv[0], &size)) Badarg(6004,"size"); if(!enif_is_list(env, argv[1])) Badarg(6004, "weights") else { ERL_NIF_TERM weights_l, weights_h, weights_t; - std::vector <GLdouble> weights_vec; GLdouble weights_tmp; weights_l = argv[1]; while(enif_get_list_cell(env, weights_l, &weights_h, &weights_t)) { @@ -14842,11 +14842,11 @@ void ecb_glWeightubvARB(ErlNifEnv* env, ErlNifPid *self, ERL_NIF_TERM argv[]) { GLint size; GLubyte *weights; + std::vector <GLubyte> weights_vec; if(!enif_get_int(env, argv[0], &size)) Badarg(6005,"size"); if(!enif_is_list(env, argv[1])) Badarg(6005, "weights") else { ERL_NIF_TERM weights_l, weights_h, weights_t; - std::vector <GLubyte> weights_vec; GLubyte weights_tmp; weights_l = argv[1]; while(enif_get_list_cell(env, weights_l, &weights_h, &weights_t)) { @@ -14863,11 +14863,11 @@ void ecb_glWeightusvARB(ErlNifEnv* env, ErlNifPid *self, ERL_NIF_TERM argv[]) { GLint size; GLushort *weights; + std::vector <GLushort> weights_vec; if(!enif_get_int(env, argv[0], &size)) Badarg(6006,"size"); if(!enif_is_list(env, argv[1])) Badarg(6006, "weights") else { ERL_NIF_TERM weights_l, weights_h, weights_t; - std::vector <GLushort> weights_vec; GLushort weights_tmp; weights_l = argv[1]; while(enif_get_list_cell(env, weights_l, &weights_h, &weights_t)) { @@ -14884,11 +14884,11 @@ void ecb_glWeightuivARB(ErlNifEnv* env, ErlNifPid *self, ERL_NIF_TERM argv[]) { GLint size; GLuint *weights; + std::vector <GLuint> weights_vec; if(!enif_get_int(env, argv[0], &size)) Badarg(6007,"size"); if(!enif_is_list(env, argv[1])) Badarg(6007, "weights") else { ERL_NIF_TERM weights_l, weights_h, weights_t; - std::vector <GLuint> weights_vec; GLuint weights_tmp; weights_l = argv[1]; while(enif_get_list_cell(env, weights_l, &weights_h, &weights_t)) { diff --git a/lib/wx/c_src/wxe_impl.cpp b/lib/wx/c_src/wxe_impl.cpp index 2938fcee81..b804229610 100644 --- a/lib/wx/c_src/wxe_impl.cpp +++ b/lib/wx/c_src/wxe_impl.cpp @@ -546,8 +546,8 @@ void WxeApp::destroyMemEnv(wxeMetaCommand &Ecmd) send_msg("debug", &msg); } - // pre-pass delete all dialogs first since they might crash erlang otherwise - for(int i=1; i < memenv->next; i++) { + // pre-pass delete all dialogs and DC's first since they might crash erlang otherwise + for(int i=memenv->next-1; i > 0; i--) { wxObject * ptr = (wxObject *) memenv->ref2ptr[i]; if(ptr) { ptrMap::iterator it = ptr2ref.find(ptr); @@ -555,23 +555,22 @@ void WxeApp::destroyMemEnv(wxeMetaCommand &Ecmd) wxeRefData *refd = it->second; if(refd->alloc_in_erl && refd->type == 2) { wxDialog *win = (wxDialog *) ptr; - if(win->IsModal()) { - win->EndModal(-1); - } + if(win->IsModal()) { win->EndModal(-1); } parent = win->GetParent(); if(parent) { ptrMap::iterator parentRef = ptr2ref.find(parent); - if(parentRef == ptr2ref.end()) { - // The parent is already dead delete the parent ref - win->SetParent(NULL); - } - } - if(recurse_level > 0) { - // Delay delete until we are out of dispatch* - } else { - delete win; + // if the parent is already dead delete the parent ref + if(parentRef == ptr2ref.end()) { win->SetParent(NULL); } } - } + // Delay delete until we are out of dispatch* + if(recurse_level == 0) { delete win; } + } else if(refd->alloc_in_erl && refd->type == 8) { + if(delete_object(ptr, refd)) { + // Delete refs for leaks and non overridden allocs + delete refd; + ptr2ref.erase(it); + } + } } } } diff --git a/lib/wx/doc/src/gl.xml b/lib/wx/doc/src/gl.xml index 68e16460f6..c1bcb6270d 100644 --- a/lib/wx/doc/src/gl.xml +++ b/lib/wx/doc/src/gl.xml @@ -88,11 +88,6 @@ <p><url href="https://www.khronos.org/registry/OpenGL-Refpages/gl2.1/xhtml/glArrayElement.xml">External documentation.</url></p></desc> </func> <func> - <name name="attachObjectARB" arity="2" clause_i="1" since=""/> - <fsummary/> - <desc><p>No documentation available.</p></desc> - </func> - <func> <name name="attachShader" arity="2" clause_i="1" since=""/> <fsummary>Attaches a shader object to a program object</fsummary> <desc> @@ -119,11 +114,6 @@ <p><url href="https://www.khronos.org/registry/OpenGL-Refpages/gl4/html/glBeginConditionalRender.xhtml">External documentation.</url></p></desc> </func> <func> - <name name="endList" arity="0" clause_i="1" since=""/> - <fsummary/> - <desc><p>No documentation available.</p></desc> - </func> - <func> <name name="beginQuery" arity="2" clause_i="1" since=""/> <name name="endQuery" arity="1" clause_i="1" since=""/> <fsummary>delimit the boundaries of a query object</fsummary> @@ -134,14 +124,10 @@ </func> <func> <name name="beginQueryIndexed" arity="3" clause_i="1" since=""/> - <fsummary/> - <desc><p>No documentation available.</p></desc> - </func> - <func> <name name="endQueryIndexed" arity="2" clause_i="1" since=""/> <fsummary>delimit the boundaries of a query object on an indexed target</fsummary> <desc> - <p><seemfa marker="gl#beginQueryIndexed/3"><c>gl:beginQueryIndexed/3</c></seemfa> and <seemfa marker="gl#endQueryIndexed/2"><c>gl:endQueryIndexed/2</c></seemfa> delimit the boundaries of a query object. <c>Query</c> must be a name previously returned from a call to <seemfa marker="gl#genQueries/1"><c>gl:genQueries/1</c></seemfa>. If a query object with name <c>Id</c> does not yet exist it is created with the type determined by <c>Target</c>. <c>Target</c> must be one of <c>?GL_SAMPLES_PASSED</c>, <c>?GL_ANY_SAMPLES_PASSED</c>, <c>?GL_PRIMITIVES_GENERATED</c>, <c>?GL_TRANSFORM_FEEDBACK_PRIMITIVES_WRITTEN</c>, or <c>?GL_TIME_ELAPSED</c>. The behavior of the query object depends on its type and is as follows. </p> + <p><seemfa marker="gl#beginQueryIndexed/3"><c>gl:beginQueryIndexed/3</c></seemfa> and <seemfa marker="gl#beginQueryIndexed/3"><c>gl:endQueryIndexed/2</c></seemfa> delimit the boundaries of a query object. <c>Query</c> must be a name previously returned from a call to <seemfa marker="gl#genQueries/1"><c>gl:genQueries/1</c></seemfa>. If a query object with name <c>Id</c> does not yet exist it is created with the type determined by <c>Target</c>. <c>Target</c> must be one of <c>?GL_SAMPLES_PASSED</c>, <c>?GL_ANY_SAMPLES_PASSED</c>, <c>?GL_PRIMITIVES_GENERATED</c>, <c>?GL_TRANSFORM_FEEDBACK_PRIMITIVES_WRITTEN</c>, or <c>?GL_TIME_ELAPSED</c>. The behavior of the query object depends on its type and is as follows. </p> <p><url href="https://www.khronos.org/registry/OpenGL-Refpages/gl4/html/glBeginQueryIndexed.xhtml">External documentation.</url></p></desc> </func> @@ -163,11 +149,6 @@ <p><url href="https://www.khronos.org/registry/OpenGL-Refpages/gl4/html/glBindAttribLocation.xhtml">External documentation.</url></p></desc> </func> <func> - <name name="bindAttribLocationARB" arity="3" clause_i="1" since=""/> - <fsummary/> - <desc><p>No documentation available.</p></desc> - </func> - <func> <name name="bindBuffer" arity="2" clause_i="1" since=""/> <fsummary>bind a named buffer object</fsummary> <desc> @@ -217,8 +198,11 @@ </func> <func> <name name="bindFragDataLocationIndexed" arity="4" clause_i="1" since=""/> - <fsummary/> - <desc><p>No documentation available.</p></desc> + <fsummary>bind a user-defined varying out variable to a fragment shader color number and index</fsummary> + <desc> + <p><seemfa marker="gl#bindFragDataLocationIndexed/4"><c>gl:bindFragDataLocationIndexed/4</c></seemfa> specifies that the varying out variable <c>Name</c> in <c>Program</c> should be bound to fragment color <c>ColorNumber</c> when the program is next linked. <c>Index</c> may be zero or one to specify that the color be used as either the first or second color input to the blend equation, respectively. </p> + + <p><url href="https://www.khronos.org/registry/OpenGL-Refpages/gl4/html/glBindFragDataLocationIndexed.xhtml">External documentation.</url></p></desc> </func> <func> <name name="bindFramebuffer" arity="2" clause_i="1" since=""/> @@ -245,11 +229,6 @@ <p><url href="https://www.khronos.org/registry/OpenGL-Refpages/gl4/html/glBindImageTextures.xhtml">External documentation.</url></p></desc> </func> <func> - <name name="bindProgramARB" arity="2" clause_i="1" since=""/> - <fsummary/> - <desc><p>No documentation available.</p></desc> - </func> - <func> <name name="bindProgramPipeline" arity="1" clause_i="1" since=""/> <fsummary>bind a program pipeline to the current context</fsummary> <desc> @@ -323,17 +302,19 @@ </func> <func> <name name="bindVertexBuffer" arity="4" clause_i="1" since=""/> + <name name="vertexArrayVertexBuffer" arity="5" clause_i="1" since=""/> <fsummary>bind a buffer to a vertex buffer bind point</fsummary> <desc> - <p><seemfa marker="gl#bindVertexBuffer/4"><c>gl:bindVertexBuffer/4</c></seemfa> and <seemfa marker="gl#vertexArrayVertexBuffer/5"><c>gl:vertexArrayVertexBuffer/5</c></seemfa> bind the buffer named <c>Buffer</c> to the vertex buffer binding point whose index is given by <c>Bindingindex</c>. <seemfa marker="gl#bindVertexBuffer/4"><c>gl:bindVertexBuffer/4</c></seemfa> modifies the binding of the currently bound vertex array object, whereas <seemfa marker="gl#vertexArrayVertexBuffer/5"><c>gl:vertexArrayVertexBuffer/5</c></seemfa> allows the caller to specify ID of the vertex array object with an argument named <c>Vaobj</c>, for which the binding should be modified. <c>Offset</c> and <c>Stride</c> specify the offset of the first element within the buffer and the distance between elements within the buffer, respectively, and are both measured in basic machine units. <c>Bindingindex</c> must be less than the value of <c>?GL_MAX_VERTEX_ATTRIB_BINDINGS</c>. <c>Offset</c> and <c>Stride</c> must be greater than or equal to zero. If <c>Buffer</c> is zero, then any buffer currently bound to the specified binding point is unbound.</p> + <p><seemfa marker="gl#bindVertexBuffer/4"><c>gl:bindVertexBuffer/4</c></seemfa> and <seemfa marker="gl#bindVertexBuffer/4"><c>gl:vertexArrayVertexBuffer/5</c></seemfa> bind the buffer named <c>Buffer</c> to the vertex buffer binding point whose index is given by <c>Bindingindex</c>. <seemfa marker="gl#bindVertexBuffer/4"><c>gl:bindVertexBuffer/4</c></seemfa> modifies the binding of the currently bound vertex array object, whereas <seemfa marker="gl#bindVertexBuffer/4"><c>gl:vertexArrayVertexBuffer/5</c></seemfa> allows the caller to specify ID of the vertex array object with an argument named <c>Vaobj</c>, for which the binding should be modified. <c>Offset</c> and <c>Stride</c> specify the offset of the first element within the buffer and the distance between elements within the buffer, respectively, and are both measured in basic machine units. <c>Bindingindex</c> must be less than the value of <c>?GL_MAX_VERTEX_ATTRIB_BINDINGS</c>. <c>Offset</c> and <c>Stride</c> must be greater than or equal to zero. If <c>Buffer</c> is zero, then any buffer currently bound to the specified binding point is unbound.</p> <p><url href="https://www.khronos.org/registry/OpenGL-Refpages/gl4/html/glBindVertexBuffer.xhtml">External documentation.</url></p></desc> </func> <func> <name name="bindVertexBuffers" arity="4" clause_i="1" since=""/> + <name name="vertexArrayVertexBuffers" arity="5" clause_i="1" since=""/> <fsummary>attach multiple buffer objects to a vertex array object</fsummary> <desc> - <p><seemfa marker="gl#bindVertexBuffers/4"><c>gl:bindVertexBuffers/4</c></seemfa> and <seemfa marker="gl#vertexArrayVertexBuffers/5"><c>gl:vertexArrayVertexBuffers/5</c></seemfa> bind storage from an array of existing buffer objects to a specified number of consecutive vertex buffer binding points units in a vertex array object. For <seemfa marker="gl#bindVertexBuffers/4"><c>gl:bindVertexBuffers/4</c></seemfa>, the vertex array object is the currently bound vertex array object. For <seemfa marker="gl#vertexArrayVertexBuffers/5"><c>gl:vertexArrayVertexBuffers/5</c></seemfa>, <c>Vaobj</c> is the name of the vertex array object. </p> + <p><seemfa marker="gl#bindVertexBuffers/4"><c>gl:bindVertexBuffers/4</c></seemfa> and <seemfa marker="gl#bindVertexBuffers/4"><c>gl:vertexArrayVertexBuffers/5</c></seemfa> bind storage from an array of existing buffer objects to a specified number of consecutive vertex buffer binding points units in a vertex array object. For <seemfa marker="gl#bindVertexBuffers/4"><c>gl:bindVertexBuffers/4</c></seemfa>, the vertex array object is the currently bound vertex array object. For <seemfa marker="gl#bindVertexBuffers/4"><c>gl:vertexArrayVertexBuffers/5</c></seemfa>, <c>Vaobj</c> is the name of the vertex array object. </p> <p><url href="https://www.khronos.org/registry/OpenGL-Refpages/gl4/html/glBindVertexBuffers.xhtml">External documentation.</url></p></desc> </func> @@ -346,11 +327,6 @@ <p><url href="https://www.khronos.org/registry/OpenGL-Refpages/gl2.1/xhtml/glBitmap.xml">External documentation.</url></p></desc> </func> <func> - <name name="blendBarrierKHR" arity="0" clause_i="1" since=""/> - <fsummary/> - <desc><p>No documentation available.</p></desc> - </func> - <func> <name name="blendColor" arity="4" clause_i="1" since=""/> <fsummary>set the blend color</fsummary> <desc> @@ -378,6 +354,7 @@ </func> <func> <name name="blendFunc" arity="2" clause_i="1" since=""/> + <name name="blendFunci" arity="3" clause_i="1" since=""/> <fsummary>specify pixel arithmetic</fsummary> <desc> <p> Pixels can be drawn using a function that blends the incoming (source) RGBA values with the RGBA values that are already in the frame buffer (the destination values). Blending is initially disabled. Use <seemfa marker="gl#enable/1"><c>gl:enable/1</c></seemfa> and <seemfa marker="gl#enable/1"><c>gl:disable/1</c></seemfa> with argument <c>?GL_BLEND</c> to enable and disable blending. </p> @@ -394,11 +371,6 @@ <p><url href="https://www.khronos.org/registry/OpenGL-Refpages/gl4/html/glBlendFuncSeparate.xhtml">External documentation.</url></p></desc> </func> <func> - <name name="blendFunci" arity="3" clause_i="1" since=""/> - <fsummary/> - <desc><p>No documentation available.</p></desc> - </func> - <func> <name name="blitFramebuffer" arity="10" clause_i="1" since=""/> <fsummary>copy a block of pixels from one framebuffer object to another</fsummary> <desc> @@ -415,11 +387,6 @@ <p><url href="https://www.khronos.org/registry/OpenGL-Refpages/gl4/html/glBufferData.xhtml">External documentation.</url></p></desc> </func> <func> - <name name="bufferPageCommitmentARB" arity="4" clause_i="1" since=""/> - <fsummary/> - <desc><p>No documentation available.</p></desc> - </func> - <func> <name name="bufferStorage" arity="4" clause_i="1" since=""/> <fsummary>creates and initializes a buffer object's immutable data store</fsummary> <desc> @@ -484,37 +451,19 @@ <p><url href="https://www.khronos.org/registry/OpenGL-Refpages/gl2.1/xhtml/glClearAccum.xml">External documentation.</url></p></desc> </func> <func> + <name name="clearBufferData" arity="5" clause_i="1" since=""/> + <name name="clearBufferSubData" arity="7" clause_i="1" since=""/> + <name name="clearBufferfi" arity="4" clause_i="1" since=""/> <name name="clearBufferfv" arity="3" clause_i="1" since=""/> <name name="clearBufferiv" arity="3" clause_i="1" since=""/> <name name="clearBufferuiv" arity="3" clause_i="1" since=""/> <fsummary>clear individual buffers of a framebuffer</fsummary> <desc> - <p> These commands clear a specified buffer of a framebuffer to specified value(s). For <c>glClearBuffer*</c>, the framebuffer is the currently bound draw framebuffer object. For <c>glClearNamedFramebuffer*</c>, <c>Framebuffer</c> is zero, indicating the default draw framebuffer, or the name of a framebuffer object. </p> + <p> These commands clear a specified buffer of a framebuffer to specified value(s). For <seemfa marker="gl#clearBufferiv/3"><c>gl:clearBuffer*()</c></seemfa>, the framebuffer is the currently bound draw framebuffer object. For <c>glClearNamedFramebuffer*</c>, <c>Framebuffer</c> is zero, indicating the default draw framebuffer, or the name of a framebuffer object. </p> <p><url href="https://www.khronos.org/registry/OpenGL-Refpages/gl4/html/glClearBuffer.xhtml">External documentation.</url></p></desc> </func> <func> - <name name="clearBufferData" arity="5" clause_i="1" since=""/> - <fsummary>fill a buffer object's data store with a fixed value</fsummary> - <desc> - <p><seemfa marker="gl#clearBufferData/5"><c>gl:clearBufferData/5</c></seemfa> and <c>glClearNamedBufferData</c> fill the entirety of a buffer object's data store with data from client memory. </p> - - <p><url href="https://www.khronos.org/registry/OpenGL-Refpages/gl4/html/glClearBufferData.xhtml">External documentation.</url></p></desc> - </func> - <func> - <name name="clearBufferSubData" arity="7" clause_i="1" since=""/> - <fsummary>fill all or part of buffer object's data store with a fixed value</fsummary> - <desc> - <p><seemfa marker="gl#clearBufferSubData/7"><c>gl:clearBufferSubData/7</c></seemfa> and <c>glClearNamedBufferSubData</c> fill a specified region of a buffer object's data store with data from client memory. </p> - - <p><url href="https://www.khronos.org/registry/OpenGL-Refpages/gl4/html/glClearBufferSubData.xhtml">External documentation.</url></p></desc> - </func> - <func> - <name name="clearBufferfi" arity="4" clause_i="1" since=""/> - <fsummary/> - <desc><p>No documentation available.</p></desc> - </func> - <func> <name name="clearColor" arity="4" clause_i="1" since=""/> <fsummary>specify clear values for the color buffers</fsummary> <desc> @@ -524,6 +473,7 @@ </func> <func> <name name="clearDepth" arity="1" clause_i="1" since=""/> + <name name="clearDepthf" arity="1" clause_i="1" since=""/> <fsummary>specify the clear value for the depth buffer</fsummary> <desc> <p><seemfa marker="gl#clearDepth/1"><c>gl:clearDepth/1</c></seemfa> specifies the depth value used by <seemfa marker="gl#clear/1"><c>gl:clear/1</c></seemfa> to clear the depth buffer. Values specified by <seemfa marker="gl#clearDepth/1"><c>gl:clearDepth/1</c></seemfa> are clamped to the range [0 1]. </p> @@ -531,11 +481,6 @@ <p><url href="https://www.khronos.org/registry/OpenGL-Refpages/gl4/html/glClearDepth.xhtml">External documentation.</url></p></desc> </func> <func> - <name name="clearDepthf" arity="1" clause_i="1" since=""/> - <fsummary/> - <desc><p>No documentation available.</p></desc> - </func> - <func> <name name="clearIndex" arity="1" clause_i="1" since=""/> <fsummary>specify the clear value for the color index buffers</fsummary> <desc> @@ -640,18 +585,14 @@ </func> <func> <name name="colorMask" arity="4" clause_i="1" since=""/> + <name name="colorMaski" arity="5" clause_i="1" since=""/> <fsummary>enable and disable writing of frame buffer color components</fsummary> <desc> - <p><seemfa marker="gl#colorMask/4"><c>gl:colorMask/4</c></seemfa> and <seemfa marker="gl#colorMaski/5"><c>gl:colorMaski/5</c></seemfa> specify whether the individual color components in the frame buffer can or cannot be written. <seemfa marker="gl#colorMaski/5"><c>gl:colorMaski/5</c></seemfa> sets the mask for a specific draw buffer, whereas <seemfa marker="gl#colorMask/4"><c>gl:colorMask/4</c></seemfa> sets the mask for all draw buffers. If <c>Red</c> is <c>?GL_FALSE</c>, for example, no change is made to the red component of any pixel in any of the color buffers, regardless of the drawing operation attempted. </p> + <p><seemfa marker="gl#colorMask/4"><c>gl:colorMask/4</c></seemfa> and <seemfa marker="gl#colorMask/4"><c>gl:colorMaski/5</c></seemfa> specify whether the individual color components in the frame buffer can or cannot be written. <seemfa marker="gl#colorMask/4"><c>gl:colorMaski/5</c></seemfa> sets the mask for a specific draw buffer, whereas <seemfa marker="gl#colorMask/4"><c>gl:colorMask/4</c></seemfa> sets the mask for all draw buffers. If <c>Red</c> is <c>?GL_FALSE</c>, for example, no change is made to the red component of any pixel in any of the color buffers, regardless of the drawing operation attempted. </p> <p><url href="https://www.khronos.org/registry/OpenGL-Refpages/gl4/html/glColorMask.xhtml">External documentation.</url></p></desc> </func> <func> - <name name="colorMaski" arity="5" clause_i="1" since=""/> - <fsummary/> - <desc><p>No documentation available.</p></desc> - </func> - <func> <name name="colorMaterial" arity="2" clause_i="1" since=""/> <fsummary>cause a material color to track the current color</fsummary> <desc> @@ -701,16 +642,6 @@ <p><url href="https://www.khronos.org/registry/OpenGL-Refpages/gl4/html/glCompileShader.xhtml">External documentation.</url></p></desc> </func> <func> - <name name="compileShaderARB" arity="1" clause_i="1" since=""/> - <fsummary/> - <desc><p>No documentation available.</p></desc> - </func> - <func> - <name name="compileShaderIncludeARB" arity="2" clause_i="1" since=""/> - <fsummary/> - <desc><p>No documentation available.</p></desc> - </func> - <func> <name name="compressedTexImage1D" arity="7" clause_i="1" since=""/> <fsummary>specify a one-dimensional texture image in a compressed format</fsummary> <desc> @@ -736,6 +667,7 @@ </func> <func> <name name="compressedTexSubImage1D" arity="7" clause_i="1" since=""/> + <name name="compressedTextureSubImage1D" arity="7" clause_i="1" since=""/> <fsummary>specify a one-dimensional texture subimage in a compressed format</fsummary> <desc> <p>Texturing allows elements of an image array to be read by shaders.</p> @@ -744,6 +676,7 @@ </func> <func> <name name="compressedTexSubImage2D" arity="9" clause_i="1" since=""/> + <name name="compressedTextureSubImage2D" arity="9" clause_i="1" since=""/> <fsummary>specify a two-dimensional texture subimage in a compressed format</fsummary> <desc> <p>Texturing allows elements of an image array to be read by shaders.</p> @@ -752,6 +685,7 @@ </func> <func> <name name="compressedTexSubImage3D" arity="11" clause_i="1" since=""/> + <name name="compressedTextureSubImage3D" arity="11" clause_i="1" since=""/> <fsummary>specify a three-dimensional texture subimage in a compressed format</fsummary> <desc> <p>Texturing allows elements of an image array to be read by shaders.</p> @@ -759,21 +693,6 @@ <p><url href="https://www.khronos.org/registry/OpenGL-Refpages/gl4/html/glCompressedTexSubImage3D.xhtml">External documentation.</url></p></desc> </func> <func> - <name name="compressedTextureSubImage1D" arity="7" clause_i="1" since=""/> - <fsummary/> - <desc><p>No documentation available.</p></desc> - </func> - <func> - <name name="compressedTextureSubImage2D" arity="9" clause_i="1" since=""/> - <fsummary/> - <desc><p>No documentation available.</p></desc> - </func> - <func> - <name name="compressedTextureSubImage3D" arity="11" clause_i="1" since=""/> - <fsummary/> - <desc><p>No documentation available.</p></desc> - </func> - <func> <name name="convolutionFilter1D" arity="6" clause_i="1" since=""/> <fsummary>define a one-dimensional convolution filter</fsummary> <desc> @@ -884,7 +803,7 @@ <name name="copyTexSubImage2D" arity="8" clause_i="1" since=""/> <fsummary>copy a two-dimensional texture subimage</fsummary> <desc> - <p><seemfa marker="gl#copyTexSubImage2D/8"><c>gl:copyTexSubImage2D/8</c></seemfa> and <c>glCopyTextureSubImage2D</c> replace a rectangular portion of a two-dimensional texture image, cube-map texture image, rectangular image, or a linear portion of a number of slices of a one-dimensional array texture with pixels from the current <c>?GL_READ_BUFFER</c> (rather than from main memory, as is the case for <seemfa marker="gl#texSubImage1D/7"><c>gl:texSubImage2D/9</c></seemfa>). </p> + <p><seemfa marker="gl#copyTexSubImage2D/8"><c>gl:copyTexSubImage2D/8</c></seemfa> and <c>glCopyTextureSubImage2D</c> replace a rectangular portion of a two-dimensional texture image, cube-map texture image, rectangular image, or a linear portion of a number of slices of a one-dimensional array texture with pixels from the current <c>?GL_READ_BUFFER</c> (rather than from main memory, as is the case for <seemfa marker="gl#texSubImage2D/9"><c>gl:texSubImage2D/9</c></seemfa>). </p> <p><url href="https://www.khronos.org/registry/OpenGL-Refpages/gl4/html/glCopyTexSubImage2D.xhtml">External documentation.</url></p></desc> </func> @@ -892,7 +811,7 @@ <name name="copyTexSubImage3D" arity="9" clause_i="1" since=""/> <fsummary>copy a three-dimensional texture subimage</fsummary> <desc> - <p><seemfa marker="gl#copyTexSubImage3D/9"><c>gl:copyTexSubImage3D/9</c></seemfa> and <c>glCopyTextureSubImage3D</c> functions replace a rectangular portion of a three-dimensional or two-dimensional array texture image with pixels from the current <c>?GL_READ_BUFFER</c> (rather than from main memory, as is the case for <seemfa marker="gl#texSubImage1D/7"><c>gl:texSubImage3D/11</c></seemfa>). </p> + <p><seemfa marker="gl#copyTexSubImage3D/9"><c>gl:copyTexSubImage3D/9</c></seemfa> and <c>glCopyTextureSubImage3D</c> functions replace a rectangular portion of a three-dimensional or two-dimensional array texture image with pixels from the current <c>?GL_READ_BUFFER</c> (rather than from main memory, as is the case for <seemfa marker="gl#texSubImage3D/11"><c>gl:texSubImage3D/11</c></seemfa>). </p> <p><url href="https://www.khronos.org/registry/OpenGL-Refpages/gl4/html/glCopyTexSubImage3D.xhtml">External documentation.</url></p></desc> </func> @@ -921,15 +840,10 @@ <p><url href="https://www.khronos.org/registry/OpenGL-Refpages/gl4/html/glCreateProgram.xhtml">External documentation.</url></p></desc> </func> <func> - <name name="createProgramObjectARB" arity="0" clause_i="1" since=""/> - <fsummary/> - <desc><p>No documentation available.</p></desc> - </func> - <func> <name name="createProgramPipelines" arity="1" clause_i="1" since=""/> <fsummary>create program pipeline objects</fsummary> <desc> - <p><seemfa marker="gl#createProgramPipelines/1"><c>gl:createProgramPipelines/1</c></seemfa> returns <c>N</c> previously unused program pipeline names in <c>Pipelines</c>, each representing a new program pipeline object initialized to the default state. </p> + <p><seemfa marker="gl#createProgramPipelines/1"><c>gl:createProgramPipelines/1</c></seemfa> returns <c>N</c> previously unused program pipeline names in <c>Pipelines</c>, each representing a new program pipeline object initialized to the default state. </p> <p><url href="https://www.khronos.org/registry/OpenGL-Refpages/gl4/html/glCreateProgramPipelines.xhtml">External documentation.</url></p></desc> </func> @@ -937,7 +851,7 @@ <name name="createQueries" arity="2" clause_i="1" since=""/> <fsummary>create query objects</fsummary> <desc> - <p><seemfa marker="gl#createQueries/2"><c>gl:createQueries/2</c></seemfa> returns <c>N</c> previously unused query object names in <c>Ids</c>, each representing a new query object with the specified <c>Target</c>. </p> + <p><seemfa marker="gl#createQueries/2"><c>gl:createQueries/2</c></seemfa> returns <c>N</c> previously unused query object names in <c>Ids</c>, each representing a new query object with the specified <c>Target</c>. </p> <p><url href="https://www.khronos.org/registry/OpenGL-Refpages/gl4/html/glCreateQueries.xhtml">External documentation.</url></p></desc> </func> @@ -945,7 +859,7 @@ <name name="createRenderbuffers" arity="1" clause_i="1" since=""/> <fsummary>create renderbuffer objects</fsummary> <desc> - <p><seemfa marker="gl#createRenderbuffers/1"><c>gl:createRenderbuffers/1</c></seemfa> returns <c>N</c> previously unused renderbuffer object names in <c>Renderbuffers</c>, each representing a new renderbuffer object initialized to the default state. </p> + <p><seemfa marker="gl#createRenderbuffers/1"><c>gl:createRenderbuffers/1</c></seemfa> returns <c>N</c> previously unused renderbuffer object names in <c>Renderbuffers</c>, each representing a new renderbuffer object initialized to the default state. </p> <p><url href="https://www.khronos.org/registry/OpenGL-Refpages/gl4/html/glCreateRenderbuffers.xhtml">External documentation.</url></p></desc> </func> @@ -953,7 +867,7 @@ <name name="createSamplers" arity="1" clause_i="1" since=""/> <fsummary>create sampler objects</fsummary> <desc> - <p><seemfa marker="gl#createSamplers/1"><c>gl:createSamplers/1</c></seemfa> returns <c>N</c> previously unused sampler names in <c>Samplers</c>, each representing a new sampler object initialized to the default state. </p> + <p><seemfa marker="gl#createSamplers/1"><c>gl:createSamplers/1</c></seemfa> returns <c>N</c> previously unused sampler names in <c>Samplers</c>, each representing a new sampler object initialized to the default state. </p> <p><url href="https://www.khronos.org/registry/OpenGL-Refpages/gl4/html/glCreateSamplers.xhtml">External documentation.</url></p></desc> </func> @@ -966,20 +880,18 @@ <p><url href="https://www.khronos.org/registry/OpenGL-Refpages/gl4/html/glCreateShader.xhtml">External documentation.</url></p></desc> </func> <func> - <name name="createShaderObjectARB" arity="1" clause_i="1" since=""/> - <fsummary/> - <desc><p>No documentation available.</p></desc> - </func> - <func> <name name="createShaderProgramv" arity="2" clause_i="1" since=""/> - <fsummary/> - <desc><p>No documentation available.</p></desc> + <fsummary>create a stand-alone program from an array of null-terminated source code strings</fsummary> + <desc> + <p><seemfa marker="gl#createShaderProgramv/2"><c>gl:createShaderProgram()</c></seemfa> creates a program object containing compiled and linked shaders for a single stage specified by <c>Type</c>. <c>Strings</c> refers to an array of <c>Count</c> strings from which to create the shader executables. </p> + + <p><url href="https://www.khronos.org/registry/OpenGL-Refpages/gl4/html/glCreateShaderProgram.xhtml">External documentation.</url></p></desc> </func> <func> <name name="createTextures" arity="2" clause_i="1" since=""/> <fsummary>create texture objects</fsummary> <desc> - <p><seemfa marker="gl#createTextures/2"><c>gl:createTextures/2</c></seemfa> returns <c>N</c> previously unused texture names in <c>Textures</c>, each representing a new texture object of the dimensionality and type specified by <c>Target</c> and initialized to the default values for that texture type. </p> + <p><seemfa marker="gl#createTextures/2"><c>gl:createTextures/2</c></seemfa> returns <c>N</c> previously unused texture names in <c>Textures</c>, each representing a new texture object of the dimensionality and type specified by <c>Target</c> and initialized to the default values for that texture type. </p> <p><url href="https://www.khronos.org/registry/OpenGL-Refpages/gl4/html/glCreateTextures.xhtml">External documentation.</url></p></desc> </func> @@ -987,7 +899,7 @@ <name name="createTransformFeedbacks" arity="1" clause_i="1" since=""/> <fsummary>create transform feedback objects</fsummary> <desc> - <p><seemfa marker="gl#createTransformFeedbacks/1"><c>gl:createTransformFeedbacks/1</c></seemfa> returns <c>N</c> previously unused transform feedback object names in <c>Ids</c>, each representing a new transform feedback object initialized to the default state. </p> + <p><seemfa marker="gl#createTransformFeedbacks/1"><c>gl:createTransformFeedbacks/1</c></seemfa> returns <c>N</c> previously unused transform feedback object names in <c>Ids</c>, each representing a new transform feedback object initialized to the default state. </p> <p><url href="https://www.khronos.org/registry/OpenGL-Refpages/gl4/html/glCreateTransformFeedbacks.xhtml">External documentation.</url></p></desc> </func> @@ -995,7 +907,7 @@ <name name="createVertexArrays" arity="1" clause_i="1" since=""/> <fsummary>create vertex array objects</fsummary> <desc> - <p><seemfa marker="gl#createVertexArrays/1"><c>gl:createVertexArrays/1</c></seemfa> returns <c>N</c> previously unused vertex array object names in <c>Arrays</c>, each representing a new vertex array object initialized to the default state. </p> + <p><seemfa marker="gl#createVertexArrays/1"><c>gl:createVertexArrays/1</c></seemfa> returns <c>N</c> previously unused vertex array object names in <c>Arrays</c>, each representing a new vertex array object initialized to the default state. </p> <p><url href="https://www.khronos.org/registry/OpenGL-Refpages/gl4/html/glCreateVertexArrays.xhtml">External documentation.</url></p></desc> </func> @@ -1008,11 +920,6 @@ <p><url href="https://www.khronos.org/registry/OpenGL-Refpages/gl4/html/glCullFace.xhtml">External documentation.</url></p></desc> </func> <func> - <name name="currentPaletteMatrixARB" arity="1" clause_i="1" since=""/> - <fsummary/> - <desc><p>No documentation available.</p></desc> - </func> - <func> <name name="debugMessageControl" arity="5" clause_i="1" since=""/> <fsummary>control the reporting of debug messages in a debug context</fsummary> <desc> @@ -1053,16 +960,6 @@ <p><url href="https://www.khronos.org/registry/OpenGL-Refpages/gl2.1/xhtml/glDeleteLists.xml">External documentation.</url></p></desc> </func> <func> - <name name="deleteNamedStringARB" arity="1" clause_i="1" since=""/> - <fsummary/> - <desc><p>No documentation available.</p></desc> - </func> - <func> - <name name="deleteObjectARB" arity="1" clause_i="1" since=""/> - <fsummary/> - <desc><p>No documentation available.</p></desc> - </func> - <func> <name name="deleteProgram" arity="1" clause_i="1" since=""/> <fsummary>Deletes a program object</fsummary> <desc> @@ -1079,11 +976,6 @@ <p><url href="https://www.khronos.org/registry/OpenGL-Refpages/gl4/html/glDeleteProgramPipelines.xhtml">External documentation.</url></p></desc> </func> <func> - <name name="deleteProgramsARB" arity="1" clause_i="1" since=""/> - <fsummary/> - <desc><p>No documentation available.</p></desc> - </func> - <func> <name name="deleteQueries" arity="1" clause_i="1" since=""/> <fsummary>delete named query objects</fsummary> <desc> @@ -1148,11 +1040,6 @@ <p><url href="https://www.khronos.org/registry/OpenGL-Refpages/gl4/html/glDeleteVertexArrays.xhtml">External documentation.</url></p></desc> </func> <func> - <name name="depthBoundsEXT" arity="2" clause_i="1" since=""/> - <fsummary/> - <desc><p>No documentation available.</p></desc> - </func> - <func> <name name="depthFunc" arity="1" clause_i="1" since=""/> <fsummary>specify the value used for depth buffer comparisons</fsummary> <desc> @@ -1179,18 +1066,19 @@ </func> <func> <name name="depthRangeArrayv" arity="2" clause_i="1" since=""/> - <fsummary/> - <desc><p>No documentation available.</p></desc> + <fsummary>specify mapping of depth values from normalized device coordinates to window coordinates for a specified set of viewports</fsummary> + <desc> + <p> After clipping and division by <c>w</c>, depth coordinates range from -1 to 1, corresponding to the near and far clipping planes. Each viewport has an independent depth range specified as a linear mapping of the normalized depth coordinates in this range to window depth coordinates. Regardless of the actual depth buffer implementation, window coordinate depth values are treated as though they range from 0 through 1 (like color components). <seemfa marker="gl#depthRangeArrayv/2"><c>gl:depthRangeArray()</c></seemfa> specifies a linear mapping of the normalized depth coordinates in this range to window depth coordinates for each viewport in the range [<c>First</c>, <c>First</c> + <c>Count</c>). Thus, the values accepted by <seemfa marker="gl#depthRangeArrayv/2"><c>gl:depthRangeArray()</c></seemfa> are both clamped to this range before they are accepted. </p> + + <p><url href="https://www.khronos.org/registry/OpenGL-Refpages/gl4/html/glDepthRangeArray.xhtml">External documentation.</url></p></desc> </func> <func> <name name="depthRangeIndexed" arity="3" clause_i="1" since=""/> - <fsummary/> - <desc><p>No documentation available.</p></desc> - </func> - <func> - <name name="detachObjectARB" arity="2" clause_i="1" since=""/> - <fsummary/> - <desc><p>No documentation available.</p></desc> + <fsummary>specify mapping of depth values from normalized device coordinates to window coordinates for a specified viewport</fsummary> + <desc> + <p> After clipping and division by <c>w</c>, depth coordinates range from -1 to 1, corresponding to the near and far clipping planes. Each viewport has an independent depth range specified as a linear mapping of the normalized depth coordinates in this range to window depth coordinates. Regardless of the actual depth buffer implementation, window coordinate depth values are treated as though they range from 0 through 1 (like color components). <seemfa marker="gl#depthRangeIndexed/3"><c>gl:depthRangeIndexed/3</c></seemfa> specifies a linear mapping of the normalized depth coordinates in this range to window depth coordinates for a specified viewport. Thus, the values accepted by <seemfa marker="gl#depthRangeIndexed/3"><c>gl:depthRangeIndexed/3</c></seemfa> are both clamped to this range before they are accepted. </p> + + <p><url href="https://www.khronos.org/registry/OpenGL-Refpages/gl4/html/glDepthRangeIndexed.xhtml">External documentation.</url></p></desc> </func> <func> <name name="detachShader" arity="2" clause_i="1" since=""/> @@ -1209,11 +1097,6 @@ <p><url href="https://www.khronos.org/registry/OpenGL-Refpages/gl4/html/glDispatchCompute.xhtml">External documentation.</url></p></desc> </func> <func> - <name name="dispatchComputeGroupSizeARB" arity="6" clause_i="1" since=""/> - <fsummary/> - <desc><p>No documentation available.</p></desc> - </func> - <func> <name name="dispatchComputeIndirect" arity="1" clause_i="1" since=""/> <fsummary>launch one or more compute work groups using parameters stored in a buffer</fsummary> <desc> @@ -1239,8 +1122,11 @@ </func> <func> <name name="drawArraysInstanced" arity="4" clause_i="1" since=""/> - <fsummary/> - <desc><p>No documentation available.</p></desc> + <fsummary>draw multiple instances of a range of elements</fsummary> + <desc> + <p><seemfa marker="gl#drawArraysInstanced/4"><c>gl:drawArraysInstanced/4</c></seemfa> behaves identically to <seemfa marker="gl#drawArrays/3"><c>gl:drawArrays/3</c></seemfa> except that <c>Instancecount</c> instances of the range of elements are executed and the value of the internal counter <c>InstanceID</c> advances for each iteration. <c>InstanceID</c> is an internal 32-bit integer counter that may be read by a vertex shader as <c>?gl_InstanceID</c>. </p> + + <p><url href="https://www.khronos.org/registry/OpenGL-Refpages/gl4/html/glDrawArraysInstanced.xhtml">External documentation.</url></p></desc> </func> <func> <name name="drawArraysInstancedBaseInstance" arity="5" clause_i="1" since=""/> @@ -1292,8 +1178,11 @@ </func> <func> <name name="drawElementsInstanced" arity="5" clause_i="1" since=""/> - <fsummary/> - <desc><p>No documentation available.</p></desc> + <fsummary>draw multiple instances of a set of elements</fsummary> + <desc> + <p><seemfa marker="gl#drawElementsInstanced/5"><c>gl:drawElementsInstanced/5</c></seemfa> behaves identically to <seemfa marker="gl#drawElements/4"><c>gl:drawElements/4</c></seemfa> except that <c>Instancecount</c> instances of the set of elements are executed and the value of the internal counter <c>InstanceID</c> advances for each iteration. <c>InstanceID</c> is an internal 32-bit integer counter that may be read by a vertex shader as <c>?gl_InstanceID</c>. </p> + + <p><url href="https://www.khronos.org/registry/OpenGL-Refpages/gl4/html/glDrawElementsInstanced.xhtml">External documentation.</url></p></desc> </func> <func> <name name="drawElementsInstancedBaseInstance" arity="6" clause_i="1" since=""/> @@ -1353,8 +1242,11 @@ </func> <func> <name name="drawTransformFeedbackInstanced" arity="3" clause_i="1" since=""/> - <fsummary/> - <desc><p>No documentation available.</p></desc> + <fsummary>render multiple instances of primitives using a count derived from a transform feedback object</fsummary> + <desc> + <p><seemfa marker="gl#drawTransformFeedbackInstanced/3"><c>gl:drawTransformFeedbackInstanced/3</c></seemfa> draws multiple copies of a range of primitives of a type specified by <c>Mode</c> using a count retrieved from the transform feedback stream specified by <c>Stream</c> of the transform feedback object specified by <c>Id</c>. Calling <seemfa marker="gl#drawTransformFeedbackInstanced/3"><c>gl:drawTransformFeedbackInstanced/3</c></seemfa> is equivalent to calling <seemfa marker="gl#drawArraysInstanced/4"><c>gl:drawArraysInstanced/4</c></seemfa> with <c>Mode</c> and <c>Instancecount</c> as specified, <c>First</c> set to zero, and <c>Count</c> set to the number of vertices captured on vertex stream zero the last time transform feedback was active on the transform feedback object named by <c>Id</c>. </p> + + <p><url href="https://www.khronos.org/registry/OpenGL-Refpages/gl4/html/glDrawTransformFeedbackInstanced.xhtml">External documentation.</url></p></desc> </func> <func> <name name="drawTransformFeedbackStream" arity="3" clause_i="1" since=""/> @@ -1366,11 +1258,15 @@ </func> <func> <name name="drawTransformFeedbackStreamInstanced" arity="4" clause_i="1" since=""/> - <fsummary/> - <desc><p>No documentation available.</p></desc> + <fsummary>render multiple instances of primitives using a count derived from a specifed stream of a transform feedback object</fsummary> + <desc> + <p><seemfa marker="gl#drawTransformFeedbackStreamInstanced/4"><c>gl:drawTransformFeedbackStreamInstanced/4</c></seemfa> draws multiple copies of a range of primitives of a type specified by <c>Mode</c> using a count retrieved from the transform feedback stream specified by <c>Stream</c> of the transform feedback object specified by <c>Id</c>. Calling <seemfa marker="gl#drawTransformFeedbackStreamInstanced/4"><c>gl:drawTransformFeedbackStreamInstanced/4</c></seemfa> is equivalent to calling <seemfa marker="gl#drawArraysInstanced/4"><c>gl:drawArraysInstanced/4</c></seemfa> with <c>Mode</c> and <c>Instancecount</c> as specified, <c>First</c> set to zero, and <c>Count</c> set to the number of vertices captured on vertex stream <c>Stream</c> the last time transform feedback was active on the transform feedback object named by <c>Id</c>. </p> + + <p><url href="https://www.khronos.org/registry/OpenGL-Refpages/gl4/html/glDrawTransformFeedbackStreamInstanced.xhtml">External documentation.</url></p></desc> </func> <func> <name name="edgeFlag" arity="1" clause_i="1" since=""/> + <name name="edgeFlagv" arity="1" clause_i="1" since=""/> <fsummary>flag edges as either boundary or nonboundary</fsummary> <desc> <p> Each vertex of a polygon, separate triangle, or separate quadrilateral specified between a <seemfa marker="gl#'begin'/1"><c>gl:'begin'/1</c></seemfa>/<seemfa marker="gl#'begin'/1"><c>gl:'end'/0</c></seemfa> pair is marked as the start of either a boundary or nonboundary edge. If the current edge flag is true when the vertex is specified, the vertex is marked as the start of a boundary edge. Otherwise, the vertex is marked as the start of a nonboundary edge. <seemfa marker="gl#edgeFlag/1"><c>gl:edgeFlag/1</c></seemfa> sets the edge flag bit to <c>?GL_TRUE</c> if <c>Flag</c> is <c>?GL_TRUE</c> and to <c>?GL_FALSE</c> otherwise. </p> @@ -1386,12 +1282,8 @@ <p><url href="https://www.khronos.org/registry/OpenGL-Refpages/gl2.1/xhtml/glEdgeFlagPointer.xml">External documentation.</url></p></desc> </func> <func> - <name name="edgeFlagv" arity="1" clause_i="1" since=""/> - <fsummary/> - <desc><p>No documentation available.</p></desc> - </func> - <func> <name name="disable" arity="1" clause_i="1" since=""/> + <name name="disablei" arity="2" clause_i="1" since=""/> <name name="enable" arity="1" clause_i="1" since=""/> <name name="enablei" arity="2" clause_i="1" since=""/> <fsummary>enable or disable server-side GL capabilities</fsummary> @@ -1411,25 +1303,16 @@ </func> <func> <name name="disableVertexArrayAttrib" arity="2" clause_i="1" since=""/> - <name name="enableVertexArrayAttrib" arity="2" clause_i="1" since=""/> - <fsummary/> - <desc><p>No documentation available.</p></desc> - </func> - <func> <name name="disableVertexAttribArray" arity="1" clause_i="1" since=""/> + <name name="enableVertexArrayAttrib" arity="2" clause_i="1" since=""/> <name name="enableVertexAttribArray" arity="1" clause_i="1" since=""/> <fsummary>Enable or disable a generic vertex attribute array</fsummary> <desc> - <p><seemfa marker="gl#enableVertexAttribArray/1"><c>gl:enableVertexAttribArray/1</c></seemfa> and <seemfa marker="gl#enableVertexArrayAttrib/2"><c>gl:enableVertexArrayAttrib/2</c></seemfa> enable the generic vertex attribute array specified by <c>Index</c>. <seemfa marker="gl#enableVertexAttribArray/1"><c>gl:enableVertexAttribArray/1</c></seemfa> uses currently bound vertex array object for the operation, whereas <seemfa marker="gl#enableVertexArrayAttrib/2"><c>gl:enableVertexArrayAttrib/2</c></seemfa> updates state of the vertex array object with ID <c>Vaobj</c>.</p> + <p><seemfa marker="gl#enableVertexAttribArray/1"><c>gl:enableVertexAttribArray/1</c></seemfa> and <seemfa marker="gl#disableVertexAttribArray/1"><c>gl:enableVertexArrayAttrib/2</c></seemfa> enable the generic vertex attribute array specified by <c>Index</c>. <seemfa marker="gl#enableVertexAttribArray/1"><c>gl:enableVertexAttribArray/1</c></seemfa> uses currently bound vertex array object for the operation, whereas <seemfa marker="gl#disableVertexAttribArray/1"><c>gl:enableVertexArrayAttrib/2</c></seemfa> updates state of the vertex array object with ID <c>Vaobj</c>.</p> <p><url href="https://www.khronos.org/registry/OpenGL-Refpages/gl4/html/glEnableVertexAttribArray.xhtml">External documentation.</url></p></desc> </func> <func> - <name name="disablei" arity="2" clause_i="1" since=""/> - <fsummary/> - <desc><p>No documentation available.</p></desc> - </func> - <func> <name name="evalCoord1d" arity="1" clause_i="1" since=""/> <name name="evalCoord1dv" arity="1" clause_i="1" since=""/> <name name="evalCoord1f" arity="1" clause_i="1" since=""/> @@ -1440,7 +1323,7 @@ <name name="evalCoord2fv" arity="1" clause_i="1" since=""/> <fsummary>evaluate enabled one- and two-dimensional maps</fsummary> <desc> - <p><seemfa marker="gl#evalCoord1d/1"><c>gl:evalCoord1()</c></seemfa> evaluates enabled one-dimensional maps at argument <c>U</c>. <seemfa marker="gl#evalCoord1d/1"><c>gl:evalCoord2()</c></seemfa> does the same for two-dimensional maps using two domain values, <c>U</c> and <c>V</c>. To define a map, call <seemfa marker="gl#map1d/6"><c>gl:map1()</c></seemfa> and <seemfa marker="gl#map1d/6"><c>gl:map2()</c></seemfa>; to enable and disable it, call <seemfa marker="gl#enable/1"><c>gl:enable/1</c></seemfa> and <seemfa marker="gl#enable/1"><c>gl:disable/1</c></seemfa>. </p> + <p><seemfa marker="gl#evalCoord1d/1"><c>gl:evalCoord1()</c></seemfa> evaluates enabled one-dimensional maps at argument <c>U</c>. <seemfa marker="gl#evalCoord1d/1"><c>gl:evalCoord2()</c></seemfa> does the same for two-dimensional maps using two domain values, <c>U</c> and <c>V</c>. To define a map, call <c>glMap1</c> and <c>glMap2</c>; to enable and disable it, call <seemfa marker="gl#enable/1"><c>gl:enable/1</c></seemfa> and <seemfa marker="gl#enable/1"><c>gl:disable/1</c></seemfa>. </p> <p><url href="https://www.khronos.org/registry/OpenGL-Refpages/gl2.1/xhtml/glEvalCoord.xml">External documentation.</url></p></desc> </func> @@ -1449,7 +1332,7 @@ <name name="evalMesh2" arity="5" clause_i="1" since=""/> <fsummary>compute a one- or two-dimensional grid of points or lines</fsummary> <desc> - <p><seemfa marker="gl#mapGrid1d/3"><c>gl:mapGrid()</c></seemfa> and <seemfa marker="gl#evalMesh1/3"><c>gl:evalMesh()</c></seemfa> are used in tandem to efficiently generate and evaluate a series of evenly-spaced map domain values. <seemfa marker="gl#evalMesh1/3"><c>gl:evalMesh()</c></seemfa> steps through the integer domain of a one- or two-dimensional grid, whose range is the domain of the evaluation maps specified by <seemfa marker="gl#map1d/6"><c>gl:map1()</c></seemfa> and <seemfa marker="gl#map1d/6"><c>gl:map2()</c></seemfa>. <c>Mode</c> determines whether the resulting vertices are connected as points, lines, or filled polygons. </p> + <p><seemfa marker="gl#mapGrid1d/3"><c>gl:mapGrid()</c></seemfa> and <seemfa marker="gl#evalMesh1/3"><c>gl:evalMesh()</c></seemfa> are used in tandem to efficiently generate and evaluate a series of evenly-spaced map domain values. <seemfa marker="gl#evalMesh1/3"><c>gl:evalMesh()</c></seemfa> steps through the integer domain of a one- or two-dimensional grid, whose range is the domain of the evaluation maps specified by <c>glMap1</c> and <c>glMap2</c>. <c>Mode</c> determines whether the resulting vertices are connected as points, lines, or filled polygons. </p> <p><url href="https://www.khronos.org/registry/OpenGL-Refpages/gl2.1/xhtml/glEvalMesh.xml">External documentation.</url></p></desc> </func> @@ -1463,11 +1346,6 @@ <p><url href="https://www.khronos.org/registry/OpenGL-Refpages/gl2.1/xhtml/glEvalPoint.xml">External documentation.</url></p></desc> </func> <func> - <name name="evaluateDepthValuesARB" arity="0" clause_i="1" since=""/> - <fsummary/> - <desc><p>No documentation available.</p></desc> - </func> - <func> <name name="feedbackBuffer" arity="3" clause_i="1" since=""/> <fsummary>controls feedback mode</fsummary> <desc> @@ -1501,6 +1379,7 @@ </func> <func> <name name="flushMappedBufferRange" arity="3" clause_i="1" since=""/> + <name name="flushMappedNamedBufferRange" arity="3" clause_i="1" since=""/> <fsummary>indicate modifications to a range of a mapped buffer</fsummary> <desc> <p><seemfa marker="gl#flushMappedBufferRange/3"><c>gl:flushMappedBufferRange/3</c></seemfa> indicates that modifications have been made to a range of a mapped buffer object. The buffer object must previously have been mapped with the <c>?GL_MAP_FLUSH_EXPLICIT_BIT</c> flag. </p> @@ -1508,11 +1387,6 @@ <p><url href="https://www.khronos.org/registry/OpenGL-Refpages/gl4/html/glFlushMappedBufferRange.xhtml">External documentation.</url></p></desc> </func> <func> - <name name="flushMappedNamedBufferRange" arity="3" clause_i="1" since=""/> - <fsummary/> - <desc><p>No documentation available.</p></desc> - </func> - <func> <name name="fogf" arity="2" clause_i="1" since=""/> <name name="fogfv" arity="2" clause_i="1" since=""/> <name name="fogi" arity="2" clause_i="1" since=""/> @@ -1544,8 +1418,11 @@ </func> <func> <name name="framebufferParameteri" arity="3" clause_i="1" since=""/> - <fsummary/> - <desc><p>No documentation available.</p></desc> + <fsummary>set a named parameter of a framebuffer object</fsummary> + <desc> + <p><seemfa marker="gl#framebufferParameteri/3"><c>gl:framebufferParameteri/3</c></seemfa> and <c>glNamedFramebufferParameteri</c> modify the value of the parameter named <c>Pname</c> in the specified framebuffer object. There are no modifiable parameters of the default draw and read framebuffer, so they are not valid targets of these commands. </p> + + <p><url href="https://www.khronos.org/registry/OpenGL-Refpages/gl4/html/glFramebufferParameteri.xhtml">External documentation.</url></p></desc> </func> <func> <name name="framebufferRenderbuffer" arity="4" clause_i="1" since=""/> @@ -1617,11 +1494,6 @@ <p><url href="https://www.khronos.org/registry/OpenGL-Refpages/gl4/html/glGenProgramPipelines.xhtml">External documentation.</url></p></desc> </func> <func> - <name name="genProgramsARB" arity="1" clause_i="1" since=""/> - <fsummary/> - <desc><p>No documentation available.</p></desc> - </func> - <func> <name name="genQueries" arity="1" clause_i="1" since=""/> <fsummary>generate query object names</fsummary> <desc> @@ -1671,18 +1543,14 @@ </func> <func> <name name="generateMipmap" arity="1" clause_i="1" since=""/> + <name name="generateTextureMipmap" arity="1" clause_i="1" since=""/> <fsummary>generate mipmaps for a specified texture object</fsummary> <desc> - <p><seemfa marker="gl#generateMipmap/1"><c>gl:generateMipmap/1</c></seemfa> and <seemfa marker="gl#generateTextureMipmap/1"><c>gl:generateTextureMipmap/1</c></seemfa> generates mipmaps for the specified texture object. For <seemfa marker="gl#generateMipmap/1"><c>gl:generateMipmap/1</c></seemfa>, the texture object that is bound to <c>Target</c>. For <seemfa marker="gl#generateTextureMipmap/1"><c>gl:generateTextureMipmap/1</c></seemfa>, <c>Texture</c> is the name of the texture object. </p> + <p><seemfa marker="gl#generateMipmap/1"><c>gl:generateMipmap/1</c></seemfa> and <seemfa marker="gl#generateMipmap/1"><c>gl:generateTextureMipmap/1</c></seemfa> generates mipmaps for the specified texture object. For <seemfa marker="gl#generateMipmap/1"><c>gl:generateMipmap/1</c></seemfa>, the texture object that is bound to <c>Target</c>. For <seemfa marker="gl#generateMipmap/1"><c>gl:generateTextureMipmap/1</c></seemfa>, <c>Texture</c> is the name of the texture object. </p> <p><url href="https://www.khronos.org/registry/OpenGL-Refpages/gl4/html/glGenerateMipmap.xhtml">External documentation.</url></p></desc> </func> <func> - <name name="generateTextureMipmap" arity="1" clause_i="1" since=""/> - <fsummary/> - <desc><p>No documentation available.</p></desc> - </func> - <func> <name name="getBooleani_v" arity="2" clause_i="1" since=""/> <name name="getBooleanv" arity="1" clause_i="1" since=""/> <name name="getDoublei_v" arity="2" clause_i="1" since=""/> @@ -1708,11 +1576,6 @@ <p><url href="https://www.khronos.org/registry/OpenGL-Refpages/gl4/html/glGetActiveAttrib.xhtml">External documentation.</url></p></desc> </func> <func> - <name name="getActiveAttribARB" arity="3" clause_i="1" since=""/> - <fsummary/> - <desc><p>No documentation available.</p></desc> - </func> - <func> <name name="getActiveSubroutineName" arity="4" clause_i="1" since=""/> <fsummary>query the name of an active shader subroutine</fsummary> <desc> @@ -1737,11 +1600,6 @@ <p><url href="https://www.khronos.org/registry/OpenGL-Refpages/gl4/html/glGetActiveUniform.xhtml">External documentation.</url></p></desc> </func> <func> - <name name="getActiveUniformARB" arity="3" clause_i="1" since=""/> - <fsummary/> - <desc><p>No documentation available.</p></desc> - </func> - <func> <name name="getActiveUniformBlockiv" arity="4" clause_i="1" since=""/> <fsummary>query information about an active uniform block</fsummary> <desc> @@ -1767,13 +1625,11 @@ </func> <func> <name name="getActiveUniformsiv" arity="3" clause_i="1" since=""/> - <fsummary/> - <desc><p>No documentation available.</p></desc> - </func> - <func> - <name name="getAttachedObjectsARB" arity="2" clause_i="1" since=""/> - <fsummary/> - <desc><p>No documentation available.</p></desc> + <fsummary>Returns information about several active uniform variables for the specified program object</fsummary> + <desc> + <p><seemfa marker="gl#getActiveUniformsiv/3"><c>gl:getActiveUniformsiv/3</c></seemfa> queries the value of the parameter named <c>Pname</c> for each of the uniforms within <c>Program</c> whose indices are specified in the array of <c>UniformCount</c> unsigned integers <c>UniformIndices</c>. Upon success, the value of the parameter for each uniform is written into the corresponding entry in the array whose address is given in <c>Params</c>. If an error is generated, nothing is written into <c>Params</c>. </p> + + <p><url href="https://www.khronos.org/registry/OpenGL-Refpages/gl4/html/glGetActiveUniformsiv.xhtml">External documentation.</url></p></desc> </func> <func> <name name="getAttachedShaders" arity="2" clause_i="1" since=""/> @@ -1792,19 +1648,13 @@ <p><url href="https://www.khronos.org/registry/OpenGL-Refpages/gl4/html/glGetAttribLocation.xhtml">External documentation.</url></p></desc> </func> <func> - <name name="getAttribLocationARB" arity="2" clause_i="1" since=""/> - <fsummary/> - <desc><p>No documentation available.</p></desc> - </func> - <func> - <name name="getBufferParameterivARB" arity="2" clause_i="1" since=""/> - <fsummary/> - <desc><p>No documentation available.</p></desc> - </func> - <func> <name name="getBufferParameteri64v" arity="2" clause_i="1" since=""/> - <fsummary/> - <desc><p>No documentation available.</p></desc> + <name name="getBufferParameterivARB" arity="2" clause_i="1" since=""/> + <fsummary>return parameters of a buffer object</fsummary> + <desc> + <p> These functions return in <c>Data</c> a selected parameter of the specified buffer object. </p> + + <p><url href="https://www.khronos.org/registry/OpenGL-Refpages/gl4/html/glGetBufferParameter.xhtml">External documentation.</url></p></desc> </func> <func> <name name="getBufferParameteriv" arity="2" clause_i="1" since=""/> @@ -1856,11 +1706,6 @@ <p><url href="https://www.khronos.org/registry/OpenGL-Refpages/gl4/html/glGetCompressedTexImage.xhtml">External documentation.</url></p></desc> </func> <func> - <name name="getCompressedTexImageARB" arity="3" clause_i="1" since=""/> - <fsummary/> - <desc><p>No documentation available.</p></desc> - </func> - <func> <name name="getConvolutionFilter" arity="4" clause_i="1" since=""/> <fsummary>get current 1D or 2D convolution filter kernel</fsummary> <desc> @@ -1934,11 +1779,6 @@ <p><url href="https://www.khronos.org/registry/OpenGL-Refpages/gl4/html/glGetGraphicsResetStatus.xhtml">External documentation.</url></p></desc> </func> <func> - <name name="getHandleARB" arity="1" clause_i="1" since=""/> - <fsummary/> - <desc><p>No documentation available.</p></desc> - </func> - <func> <name name="getHistogram" arity="5" clause_i="1" since=""/> <fsummary>get histogram table</fsummary> <desc> @@ -1956,27 +1796,13 @@ <p><url href="https://www.khronos.org/registry/OpenGL-Refpages/gl2.1/xhtml/glGetHistogramParameter.xml">External documentation.</url></p></desc> </func> <func> - <name name="getImageHandleARB" arity="5" clause_i="1" since=""/> - <fsummary/> - <desc><p>No documentation available.</p></desc> - </func> - <func> - <name name="getInfoLogARB" arity="2" clause_i="1" since=""/> - <fsummary/> - <desc><p>No documentation available.</p></desc> - </func> - <func> + <name name="getInternalformati64v" arity="4" clause_i="1" since=""/> <name name="getInternalformativ" arity="4" clause_i="1" since=""/> <fsummary>retrieve information about implementation-dependent support for internal formats</fsummary> <desc> - <p><seemfa marker="gl#getInternalformativ/4"><c>gl:getInternalformativ/4</c></seemfa> and <seemfa marker="gl#getInternalformati64v/4"><c>gl:getInternalformati64v/4</c></seemfa> retrieve information about implementation-dependent support for internal formats. <c>Target</c> indicates the target with which the internal format will be used and must be one of <c>?GL_RENDERBUFFER</c>, <c>?GL_TEXTURE_2D_MULTISAMPLE</c>, or <c>?GL_TEXTURE_2D_MULTISAMPLE_ARRAY</c>, corresponding to usage as a renderbuffer, two-dimensional multisample texture or two-dimensional multisample array texture, respectively. </p> + <p><seemfa marker="gl#getInternalformativ/4"><c>gl:getInternalformativ/4</c></seemfa> and <seemfa marker="gl#getInternalformativ/4"><c>gl:getInternalformati64v/4</c></seemfa> retrieve information about implementation-dependent support for internal formats. <c>Target</c> indicates the target with which the internal format will be used and must be one of <c>?GL_RENDERBUFFER</c>, <c>?GL_TEXTURE_2D_MULTISAMPLE</c>, or <c>?GL_TEXTURE_2D_MULTISAMPLE_ARRAY</c>, corresponding to usage as a renderbuffer, two-dimensional multisample texture or two-dimensional multisample array texture, respectively. </p> - <p><url href="https://www.khronos.org/registry/OpenGL-Refpages/gl4/html/glGetInternalformat.xhtml">External documentation.</url></p></desc> - </func> - <func> - <name name="getInternalformati64v" arity="4" clause_i="1" since=""/> - <fsummary/> - <desc><p>No documentation available.</p></desc> + <p><url href="https://www.khronos.org/registry/OpenGL-Refpages/gl4/html/glGetInternalFormat.xhtml">External documentation.</url></p></desc> </func> <func> <name name="getLightfv" arity="2" clause_i="1" since=""/> @@ -1993,7 +1819,7 @@ <name name="getMapiv" arity="3" clause_i="1" since=""/> <fsummary>return evaluator parameters</fsummary> <desc> - <p><seemfa marker="gl#map1d/6"><c>gl:map1()</c></seemfa> and <seemfa marker="gl#map1d/6"><c>gl:map2()</c></seemfa> define evaluators. <seemfa marker="gl#getMapdv/3"><c>gl:getMap()</c></seemfa> returns evaluator parameters. <c>Target</c> chooses a map, <c>Query</c> selects a specific parameter, and <c>V</c> points to storage where the values will be returned. </p> + <p><c>glMap1</c> and <c>glMap2</c> define evaluators. <seemfa marker="gl#getMapdv/3"><c>gl:getMap()</c></seemfa> returns evaluator parameters. <c>Target</c> chooses a map, <c>Query</c> selects a specific parameter, and <c>V</c> points to storage where the values will be returned. </p> <p><url href="https://www.khronos.org/registry/OpenGL-Refpages/gl2.1/xhtml/glGetMap.xml">External documentation.</url></p></desc> </func> @@ -2032,18 +1858,12 @@ <p><url href="https://www.khronos.org/registry/OpenGL-Refpages/gl4/html/glGetMultisample.xhtml">External documentation.</url></p></desc> </func> <func> - <name name="getObjectParameterfvARB" arity="2" clause_i="1" since=""/> - <name name="getObjectParameterivARB" arity="2" clause_i="1" since=""/> - <fsummary/> - <desc><p>No documentation available.</p></desc> - </func> - <func> <name name="getPixelMapfv" arity="2" clause_i="1" since=""/> <name name="getPixelMapuiv" arity="2" clause_i="1" since=""/> <name name="getPixelMapusv" arity="2" clause_i="1" since=""/> <fsummary>return the specified pixel map</fsummary> <desc> - <p> See the <seemfa marker="gl#pixelMapfv/3"><c>gl:pixelMap()</c></seemfa> reference page for a description of the acceptable values for the <c>Map</c> parameter. <seemfa marker="gl#getPixelMapfv/2"><c>gl:getPixelMap()</c></seemfa> returns in <c>Data</c> the contents of the pixel map specified in <c>Map</c>. Pixel maps are used during the execution of <seemfa marker="gl#readPixels/7"><c>gl:readPixels/7</c></seemfa>, <seemfa marker="gl#drawPixels/5"><c>gl:drawPixels/5</c></seemfa>, <seemfa marker="gl#copyPixels/5"><c>gl:copyPixels/5</c></seemfa>, <seemfa marker="gl#texImage1D/8"><c>gl:texImage1D/8</c></seemfa>, <seemfa marker="gl#texImage2D/9"><c>gl:texImage2D/9</c></seemfa>, <seemfa marker="gl#texImage3D/10"><c>gl:texImage3D/10</c></seemfa>, <seemfa marker="gl#texSubImage1D/7"><c>gl:texSubImage1D/7</c></seemfa>, <seemfa marker="gl#texSubImage1D/7"><c>gl:texSubImage2D/9</c></seemfa>, <seemfa marker="gl#texSubImage1D/7"><c>gl:texSubImage3D/11</c></seemfa>, <seemfa marker="gl#copyTexImage1D/7"><c>gl:copyTexImage1D/7</c></seemfa>, <seemfa marker="gl#copyTexImage2D/8"><c>gl:copyTexImage2D/8</c></seemfa>, <seemfa marker="gl#copyTexSubImage1D/6"><c>gl:copyTexSubImage1D/6</c></seemfa>, <seemfa marker="gl#copyTexSubImage2D/8"><c>gl:copyTexSubImage2D/8</c></seemfa>, and <seemfa marker="gl#copyTexSubImage3D/9"><c>gl:copyTexSubImage3D/9</c></seemfa>. to map color indices, stencil indices, color components, and depth components to other values. </p> + <p> See the <seemfa marker="gl#pixelMapfv/3"><c>gl:pixelMap()</c></seemfa> reference page for a description of the acceptable values for the <c>Map</c> parameter. <seemfa marker="gl#getPixelMapfv/2"><c>gl:getPixelMap()</c></seemfa> returns in <c>Data</c> the contents of the pixel map specified in <c>Map</c>. Pixel maps are used during the execution of <seemfa marker="gl#readPixels/7"><c>gl:readPixels/7</c></seemfa>, <seemfa marker="gl#drawPixels/5"><c>gl:drawPixels/5</c></seemfa>, <seemfa marker="gl#copyPixels/5"><c>gl:copyPixels/5</c></seemfa>, <seemfa marker="gl#texImage1D/8"><c>gl:texImage1D/8</c></seemfa>, <seemfa marker="gl#texImage2D/9"><c>gl:texImage2D/9</c></seemfa>, <seemfa marker="gl#texImage3D/10"><c>gl:texImage3D/10</c></seemfa>, <seemfa marker="gl#texSubImage1D/7"><c>gl:texSubImage1D/7</c></seemfa>, <seemfa marker="gl#texSubImage2D/9"><c>gl:texSubImage2D/9</c></seemfa>, <seemfa marker="gl#texSubImage3D/11"><c>gl:texSubImage3D/11</c></seemfa>, <seemfa marker="gl#copyTexImage1D/7"><c>gl:copyTexImage1D/7</c></seemfa>, <seemfa marker="gl#copyTexImage2D/8"><c>gl:copyTexImage2D/8</c></seemfa>, <seemfa marker="gl#copyTexSubImage1D/6"><c>gl:copyTexSubImage1D/6</c></seemfa>, <seemfa marker="gl#copyTexSubImage2D/8"><c>gl:copyTexSubImage2D/8</c></seemfa>, and <seemfa marker="gl#copyTexSubImage3D/9"><c>gl:copyTexSubImage3D/9</c></seemfa>. to map color indices, stencil indices, color components, and depth components to other values. </p> <p><url href="https://www.khronos.org/registry/OpenGL-Refpages/gl2.1/xhtml/glGetPixelMap.xml">External documentation.</url></p></desc> </func> @@ -2072,12 +1892,6 @@ <p><url href="https://www.khronos.org/registry/OpenGL-Refpages/gl4/html/glGetProgramBinary.xhtml">External documentation.</url></p></desc> </func> <func> - <name name="getProgramEnvParameterdvARB" arity="2" clause_i="1" since=""/> - <name name="getProgramEnvParameterfvARB" arity="2" clause_i="1" since=""/> - <fsummary/> - <desc><p>No documentation available.</p></desc> - </func> - <func> <name name="getProgramInfoLog" arity="2" clause_i="1" since=""/> <fsummary>Returns the information log for a program object</fsummary> <desc> @@ -2094,12 +1908,6 @@ <p><url href="https://www.khronos.org/registry/OpenGL-Refpages/gl4/html/glGetProgramInterface.xhtml">External documentation.</url></p></desc> </func> <func> - <name name="getProgramLocalParameterdvARB" arity="2" clause_i="1" since=""/> - <name name="getProgramLocalParameterfvARB" arity="2" clause_i="1" since=""/> - <fsummary/> - <desc><p>No documentation available.</p></desc> - </func> - <func> <name name="getProgramPipelineiv" arity="2" clause_i="1" since=""/> <fsummary>retrieve properties of a program pipeline object</fsummary> <desc> @@ -2156,32 +1964,6 @@ <p><url href="https://www.khronos.org/registry/OpenGL-Refpages/gl4/html/glGetProgramStage.xhtml">External documentation.</url></p></desc> </func> <func> - <name name="getProgramStringARB" arity="3" clause_i="1" since=""/> - <fsummary/> - <desc><p>No documentation available.</p></desc> - </func> - <func> - <name name="getQueryiv" arity="2" clause_i="1" since=""/> - <fsummary/> - <desc><p>No documentation available.</p></desc> - </func> - <func> - <name name="getQueryBufferObjectiv" arity="4" clause_i="1" since=""/> - <name name="getQueryBufferObjectuiv" arity="4" clause_i="1" since=""/> - <fsummary/> - <desc><p>No documentation available.</p></desc> - </func> - <func> - <name name="getQueryBufferObjecti64v" arity="4" clause_i="1" since=""/> - <fsummary/> - <desc><p>No documentation available.</p></desc> - </func> - <func> - <name name="getQueryBufferObjectui64v" arity="4" clause_i="1" since=""/> - <fsummary/> - <desc><p>No documentation available.</p></desc> - </func> - <func> <name name="getQueryIndexediv" arity="3" clause_i="1" since=""/> <fsummary>return parameters of an indexed query object target</fsummary> <desc> @@ -2190,23 +1972,27 @@ <p><url href="https://www.khronos.org/registry/OpenGL-Refpages/gl4/html/glGetQueryIndexed.xhtml">External documentation.</url></p></desc> </func> <func> + <name name="getQueryBufferObjecti64v" arity="4" clause_i="1" since=""/> + <name name="getQueryBufferObjectiv" arity="4" clause_i="1" since=""/> + <name name="getQueryBufferObjectui64v" arity="4" clause_i="1" since=""/> + <name name="getQueryBufferObjectuiv" arity="4" clause_i="1" since=""/> + <name name="getQueryObjecti64v" arity="2" clause_i="1" since=""/> <name name="getQueryObjectiv" arity="2" clause_i="1" since=""/> + <name name="getQueryObjectui64v" arity="2" clause_i="1" since=""/> <name name="getQueryObjectuiv" arity="2" clause_i="1" since=""/> <fsummary>return parameters of a query object</fsummary> <desc> - <p> These commands return a selected parameter of the query object specified by <c>Id</c>. <seemfa marker="gl#getQueryObjectiv/2"><c>gl:getQueryObject()</c></seemfa> returns in <c>Params</c> a selected parameter of the query object specified by <c>Id</c>. <seemfa marker="gl#getQueryBufferObjectiv/4"><c>gl:getQueryBufferObject()</c></seemfa> returns in <c>Buffer</c> a selected parameter of the query object specified by <c>Id</c>, by writing it to <c>Buffer</c>'s data store at the byte offset specified by <c>Offset</c>. </p> + <p> These commands return a selected parameter of the query object specified by <c>Id</c>. <seemfa marker="gl#getQueryObjectiv/2"><c>gl:getQueryObject()</c></seemfa> returns in <c>Params</c> a selected parameter of the query object specified by <c>Id</c>. <seemfa marker="gl#getQueryObjectiv/2"><c>gl:getQueryBufferObject()</c></seemfa> returns in <c>Buffer</c> a selected parameter of the query object specified by <c>Id</c>, by writing it to <c>Buffer</c>'s data store at the byte offset specified by <c>Offset</c>. </p> <p><url href="https://www.khronos.org/registry/OpenGL-Refpages/gl4/html/glGetQueryObject.xhtml">External documentation.</url></p></desc> </func> <func> - <name name="getQueryObjecti64v" arity="2" clause_i="1" since=""/> - <fsummary/> - <desc><p>No documentation available.</p></desc> - </func> - <func> - <name name="getQueryObjectui64v" arity="2" clause_i="1" since=""/> - <fsummary/> - <desc><p>No documentation available.</p></desc> + <name name="getQueryiv" arity="2" clause_i="1" since=""/> + <fsummary>return parameters of a query object target</fsummary> + <desc> + <p><seemfa marker="gl#getQueryiv/2"><c>gl:getQueryiv/2</c></seemfa> returns in <c>Params</c> a selected parameter of the query object target specified by <c>Target</c>. </p> + + <p><url href="https://www.khronos.org/registry/OpenGL-Refpages/gl4/html/glGetQueryiv.xhtml">External documentation.</url></p></desc> </func> <func> <name name="getRenderbufferParameteriv" arity="2" clause_i="1" since=""/> @@ -2218,6 +2004,7 @@ </func> <func> <name name="getSamplerParameterIiv" arity="2" clause_i="1" since=""/> + <name name="getSamplerParameterIuiv" arity="2" clause_i="1" since=""/> <name name="getSamplerParameterfv" arity="2" clause_i="1" since=""/> <name name="getSamplerParameteriv" arity="2" clause_i="1" since=""/> <fsummary>return sampler parameter values</fsummary> @@ -2227,11 +2014,6 @@ <p><url href="https://www.khronos.org/registry/OpenGL-Refpages/gl4/html/glGetSamplerParameter.xhtml">External documentation.</url></p></desc> </func> <func> - <name name="getSamplerParameterIuiv" arity="2" clause_i="1" since=""/> - <fsummary/> - <desc><p>No documentation available.</p></desc> - </func> - <func> <name name="getShaderiv" arity="2" clause_i="1" since=""/> <fsummary>Returns a parameter from a shader object</fsummary> <desc> @@ -2264,11 +2046,6 @@ <p><url href="https://www.khronos.org/registry/OpenGL-Refpages/gl4/html/glGetShaderSource.xhtml">External documentation.</url></p></desc> </func> <func> - <name name="getShaderSourceARB" arity="2" clause_i="1" since=""/> - <fsummary/> - <desc><p>No documentation available.</p></desc> - </func> - <func> <name name="getString" arity="1" clause_i="1" since=""/> <name name="getStringi" arity="2" clause_i="1" since=""/> <fsummary>return a string describing the current GL connection</fsummary> @@ -2306,7 +2083,7 @@ <name name="getTexEnviv" arity="2" clause_i="1" since=""/> <fsummary>return texture environment parameters</fsummary> <desc> - <p><seemfa marker="gl#getTexEnvfv/2"><c>gl:getTexEnv()</c></seemfa> returns in <c>Params</c> selected values of a texture environment that was specified with <seemfa marker="gl#texEnvfv/3"><c>gl:texEnv()</c></seemfa>. <c>Target</c> specifies a texture environment. </p> + <p><seemfa marker="gl#getTexEnvfv/2"><c>gl:getTexEnv()</c></seemfa> returns in <c>Params</c> selected values of a texture environment that was specified with <seemfa marker="gl#texEnvf/3"><c>gl:texEnv()</c></seemfa>. <c>Target</c> specifies a texture environment. </p> <p><url href="https://www.khronos.org/registry/OpenGL-Refpages/gl2.1/xhtml/glGetTexEnv.xml">External documentation.</url></p></desc> </func> @@ -2339,6 +2116,7 @@ </func> <func> <name name="getTexParameterIiv" arity="2" clause_i="1" since=""/> + <name name="getTexParameterIuiv" arity="2" clause_i="1" since=""/> <name name="getTexParameterfv" arity="2" clause_i="1" since=""/> <name name="getTexParameteriv" arity="2" clause_i="1" since=""/> <fsummary>return texture parameter values</fsummary> @@ -2348,11 +2126,6 @@ <p><url href="https://www.khronos.org/registry/OpenGL-Refpages/gl4/html/glGetTexParameter.xhtml">External documentation.</url></p></desc> </func> <func> - <name name="getTexParameterIuiv" arity="2" clause_i="1" since=""/> - <fsummary/> - <desc><p>No documentation available.</p></desc> - </func> - <func> <name name="getTransformFeedbackVarying" arity="3" clause_i="1" since=""/> <fsummary>retrieve information about varying variables selected for transform feedback</fsummary> <desc> @@ -2372,12 +2145,6 @@ <p><url href="https://www.khronos.org/registry/OpenGL-Refpages/gl4/html/glGetUniform.xhtml">External documentation.</url></p></desc> </func> <func> - <name name="getUniformfvARB" arity="2" clause_i="1" since=""/> - <name name="getUniformivARB" arity="2" clause_i="1" since=""/> - <fsummary/> - <desc><p>No documentation available.</p></desc> - </func> - <func> <name name="getUniformBlockIndex" arity="2" clause_i="1" since=""/> <fsummary>retrieve the index of a named uniform block</fsummary> <desc> @@ -2402,11 +2169,6 @@ <p><url href="https://www.khronos.org/registry/OpenGL-Refpages/gl4/html/glGetUniformLocation.xhtml">External documentation.</url></p></desc> </func> <func> - <name name="getUniformLocationARB" arity="2" clause_i="1" since=""/> - <fsummary/> - <desc><p>No documentation available.</p></desc> - </func> - <func> <name name="getUniformSubroutineuiv" arity="2" clause_i="1" since=""/> <fsummary>retrieve the value of a subroutine uniform of a given shader stage of the current program</fsummary> <desc> @@ -2415,17 +2177,9 @@ <p><url href="https://www.khronos.org/registry/OpenGL-Refpages/gl4/html/glGetUniformSubroutine.xhtml">External documentation.</url></p></desc> </func> <func> - <name name="getUniformi64vARB" arity="2" clause_i="1" since=""/> - <fsummary/> - <desc><p>No documentation available.</p></desc> - </func> - <func> - <name name="getUniformui64vARB" arity="2" clause_i="1" since=""/> - <fsummary/> - <desc><p>No documentation available.</p></desc> - </func> - <func> <name name="getVertexAttribIiv" arity="2" clause_i="1" since=""/> + <name name="getVertexAttribIuiv" arity="2" clause_i="1" since=""/> + <name name="getVertexAttribLdv" arity="2" clause_i="1" since=""/> <name name="getVertexAttribdv" arity="2" clause_i="1" since=""/> <name name="getVertexAttribfv" arity="2" clause_i="1" since=""/> <name name="getVertexAttribiv" arity="2" clause_i="1" since=""/> @@ -2436,16 +2190,6 @@ <p><url href="https://www.khronos.org/registry/OpenGL-Refpages/gl4/html/glGetVertexAttrib.xhtml">External documentation.</url></p></desc> </func> <func> - <name name="getVertexAttribIuiv" arity="2" clause_i="1" since=""/> - <fsummary/> - <desc><p>No documentation available.</p></desc> - </func> - <func> - <name name="getVertexAttribLdv" arity="2" clause_i="1" since=""/> - <fsummary/> - <desc><p>No documentation available.</p></desc> - </func> - <func> <name name="hint" arity="2" clause_i="1" since=""/> <fsummary>specify implementation-specific hints</fsummary> <desc> @@ -2554,7 +2298,7 @@ <name name="invalidateTexSubImage" arity="8" clause_i="1" since=""/> <fsummary>invalidate a region of a texture image</fsummary> <desc> - <p><seemfa marker="gl#invalidateTexSubImage/8"><c>gl:invalidateTexSubImage/8</c></seemfa> invalidates all or part of a texture image. <c>Texture</c> and <c>Level</c> indicated which texture image is being invalidated. After this command, data in that subregion have undefined values. <c>Xoffset</c>, <c>Yoffset</c>, <c>Zoffset</c>, <c>Width</c>, <c>Height</c>, and <c>Depth</c> are interpreted as they are in <seemfa marker="gl#texSubImage1D/7"><c>gl:texSubImage3D/11</c></seemfa>. For texture targets that don't have certain dimensions, this command treats those dimensions as having a size of 1. For example, to invalidate a portion of a two- dimensional texture, the application would use <c>Zoffset</c> equal to zero and <c>Depth</c> equal to one. Cube map textures are treated as an array of six slices in the z-dimension, where a value of <c>Zoffset</c> is interpreted as specifying face <c>?GL_TEXTURE_CUBE_MAP_POSITIVE_X</c> + <c>Zoffset</c>. </p> + <p><seemfa marker="gl#invalidateTexSubImage/8"><c>gl:invalidateTexSubImage/8</c></seemfa> invalidates all or part of a texture image. <c>Texture</c> and <c>Level</c> indicated which texture image is being invalidated. After this command, data in that subregion have undefined values. <c>Xoffset</c>, <c>Yoffset</c>, <c>Zoffset</c>, <c>Width</c>, <c>Height</c>, and <c>Depth</c> are interpreted as they are in <seemfa marker="gl#texSubImage3D/11"><c>gl:texSubImage3D/11</c></seemfa>. For texture targets that don't have certain dimensions, this command treats those dimensions as having a size of 1. For example, to invalidate a portion of a two- dimensional texture, the application would use <c>Zoffset</c> equal to zero and <c>Depth</c> equal to one. Cube map textures are treated as an array of six slices in the z-dimension, where a value of <c>Zoffset</c> is interpreted as specifying face <c>?GL_TEXTURE_CUBE_MAP_POSITIVE_X</c> + <c>Zoffset</c>. </p> <p><url href="https://www.khronos.org/registry/OpenGL-Refpages/gl4/html/glInvalidateTexSubImage.xhtml">External documentation.</url></p></desc> </func> @@ -2568,18 +2312,14 @@ </func> <func> <name name="isEnabled" arity="1" clause_i="1" since=""/> + <name name="isEnabledi" arity="2" clause_i="1" since=""/> <fsummary>test whether a capability is enabled</fsummary> <desc> - <p><seemfa marker="gl#isEnabled/1"><c>gl:isEnabled/1</c></seemfa> returns <c>?GL_TRUE</c> if <c>Cap</c> is an enabled capability and returns <c>?GL_FALSE</c> otherwise. Boolean states that are indexed may be tested with <seemfa marker="gl#isEnabledi/2"><c>gl:isEnabledi/2</c></seemfa>. For <seemfa marker="gl#isEnabledi/2"><c>gl:isEnabledi/2</c></seemfa>, <c>Index</c> specifies the index of the capability to test. <c>Index</c> must be between zero and the count of indexed capabilities for <c>Cap</c>. Initially all capabilities except <c>?GL_DITHER</c> are disabled; <c>?GL_DITHER</c> is initially enabled. </p> + <p><seemfa marker="gl#isEnabled/1"><c>gl:isEnabled/1</c></seemfa> returns <c>?GL_TRUE</c> if <c>Cap</c> is an enabled capability and returns <c>?GL_FALSE</c> otherwise. Boolean states that are indexed may be tested with <seemfa marker="gl#isEnabled/1"><c>gl:isEnabledi/2</c></seemfa>. For <seemfa marker="gl#isEnabled/1"><c>gl:isEnabledi/2</c></seemfa>, <c>Index</c> specifies the index of the capability to test. <c>Index</c> must be between zero and the count of indexed capabilities for <c>Cap</c>. Initially all capabilities except <c>?GL_DITHER</c> are disabled; <c>?GL_DITHER</c> is initially enabled. </p> <p><url href="https://www.khronos.org/registry/OpenGL-Refpages/gl4/html/glIsEnabled.xhtml">External documentation.</url></p></desc> </func> <func> - <name name="isEnabledi" arity="2" clause_i="1" since=""/> - <fsummary/> - <desc><p>No documentation available.</p></desc> - </func> - <func> <name name="isFramebuffer" arity="1" clause_i="1" since=""/> <fsummary>determine if a name corresponds to a framebuffer object</fsummary> <desc> @@ -2588,11 +2328,6 @@ <p><url href="https://www.khronos.org/registry/OpenGL-Refpages/gl4/html/glIsFramebuffer.xhtml">External documentation.</url></p></desc> </func> <func> - <name name="isImageHandleResidentARB" arity="1" clause_i="1" since=""/> - <fsummary/> - <desc><p>No documentation available.</p></desc> - </func> - <func> <name name="isList" arity="1" clause_i="1" since=""/> <fsummary>determine if a name corresponds to a display list</fsummary> <desc> @@ -2601,11 +2336,6 @@ <p><url href="https://www.khronos.org/registry/OpenGL-Refpages/gl2.1/xhtml/glIsList.xml">External documentation.</url></p></desc> </func> <func> - <name name="isNamedStringARB" arity="1" clause_i="1" since=""/> - <fsummary/> - <desc><p>No documentation available.</p></desc> - </func> - <func> <name name="isProgram" arity="1" clause_i="1" since=""/> <fsummary>Determines if a name corresponds to a program object</fsummary> <desc> @@ -2670,11 +2400,6 @@ <p><url href="https://www.khronos.org/registry/OpenGL-Refpages/gl4/html/glIsTexture.xhtml">External documentation.</url></p></desc> </func> <func> - <name name="isTextureHandleResidentARB" arity="1" clause_i="1" since=""/> - <fsummary/> - <desc><p>No documentation available.</p></desc> - </func> - <func> <name name="isTransformFeedback" arity="1" clause_i="1" since=""/> <fsummary>determine if a name corresponds to a transform feedback object</fsummary> <desc> @@ -2737,11 +2462,6 @@ <p><url href="https://www.khronos.org/registry/OpenGL-Refpages/gl4/html/glLinkProgram.xhtml">External documentation.</url></p></desc> </func> <func> - <name name="linkProgramARB" arity="1" clause_i="1" since=""/> - <fsummary/> - <desc><p>No documentation available.</p></desc> - </func> - <func> <name name="listBase" arity="1" clause_i="1" since=""/> <fsummary>set the display-list base for <c>glCallLists</c></fsummary> <desc> @@ -2784,12 +2504,6 @@ <p><url href="https://www.khronos.org/registry/OpenGL-Refpages/gl2.1/xhtml/glLoadTransposeMatrix.xml">External documentation.</url></p></desc> </func> <func> - <name name="loadTransposeMatrixdARB" arity="1" clause_i="1" since=""/> - <name name="loadTransposeMatrixfARB" arity="1" clause_i="1" since=""/> - <fsummary/> - <desc><p>No documentation available.</p></desc> - </func> - <func> <name name="logicOp" arity="1" clause_i="1" since=""/> <fsummary>specify a logical pixel operation for rendering</fsummary> <desc> @@ -2798,32 +2512,22 @@ <p><url href="https://www.khronos.org/registry/OpenGL-Refpages/gl4/html/glLogicOp.xhtml">External documentation.</url></p></desc> </func> <func> - <name name="makeImageHandleNonResidentARB" arity="1" clause_i="1" since=""/> - <fsummary/> - <desc><p>No documentation available.</p></desc> - </func> - <func> - <name name="makeImageHandleResidentARB" arity="2" clause_i="1" since=""/> - <fsummary/> - <desc><p>No documentation available.</p></desc> - </func> - <func> - <name name="makeTextureHandleNonResidentARB" arity="1" clause_i="1" since=""/> - <fsummary/> - <desc><p>No documentation available.</p></desc> - </func> - <func> - <name name="makeTextureHandleResidentARB" arity="1" clause_i="1" since=""/> - <fsummary/> - <desc><p>No documentation available.</p></desc> - </func> - <func> <name name="map1d" arity="6" clause_i="1" since=""/> <name name="map1f" arity="6" clause_i="1" since=""/> + <fsummary>define a one-dimensional evaluator</fsummary> + <desc> + <p> Evaluators provide a way to use polynomial or rational polynomial mapping to produce vertices, normals, texture coordinates, and colors. The values produced by an evaluator are sent to further stages of GL processing just as if they had been presented using <seemfa marker="gl#vertex2d/2"><c>gl:vertex()</c></seemfa>, <seemfa marker="gl#normal3b/3"><c>gl:normal()</c></seemfa>, <seemfa marker="gl#texCoord1d/1"><c>gl:texCoord()</c></seemfa>, and <seemfa marker="gl#color3b/3"><c>gl:color()</c></seemfa> commands, except that the generated values do not update the current normal, texture coordinates, or color. </p> + + <p><url href="https://www.khronos.org/registry/OpenGL-Refpages/gl2.1/xhtml/glMap1.xml">External documentation.</url></p></desc> + </func> + <func> <name name="map2d" arity="10" clause_i="1" since=""/> <name name="map2f" arity="10" clause_i="1" since=""/> - <fsummary/> - <desc><p>No documentation available.</p></desc> + <fsummary>define a two-dimensional evaluator</fsummary> + <desc> + <p> Evaluators provide a way to use polynomial or rational polynomial mapping to produce vertices, normals, texture coordinates, and colors. The values produced by an evaluator are sent on to further stages of GL processing just as if they had been presented using <seemfa marker="gl#vertex2d/2"><c>gl:vertex()</c></seemfa>, <seemfa marker="gl#normal3b/3"><c>gl:normal()</c></seemfa>, <seemfa marker="gl#texCoord1d/1"><c>gl:texCoord()</c></seemfa>, and <seemfa marker="gl#color3b/3"><c>gl:color()</c></seemfa> commands, except that the generated values do not update the current normal, texture coordinates, or color. </p> + + <p><url href="https://www.khronos.org/registry/OpenGL-Refpages/gl2.1/xhtml/glMap2.xml">External documentation.</url></p></desc> </func> <func> <name name="mapGrid1d" arity="3" clause_i="1" since=""/> @@ -2832,7 +2536,7 @@ <name name="mapGrid2f" arity="6" clause_i="1" since=""/> <fsummary>define a one- or two-dimensional mesh</fsummary> <desc> - <p><seemfa marker="gl#mapGrid1d/3"><c>gl:mapGrid()</c></seemfa> and <seemfa marker="gl#evalMesh1/3"><c>gl:evalMesh()</c></seemfa> are used together to efficiently generate and evaluate a series of evenly-spaced map domain values. <seemfa marker="gl#evalMesh1/3"><c>gl:evalMesh()</c></seemfa> steps through the integer domain of a one- or two-dimensional grid, whose range is the domain of the evaluation maps specified by <seemfa marker="gl#map1d/6"><c>gl:map1()</c></seemfa> and <seemfa marker="gl#map1d/6"><c>gl:map2()</c></seemfa>. </p> + <p><seemfa marker="gl#mapGrid1d/3"><c>gl:mapGrid()</c></seemfa> and <seemfa marker="gl#evalMesh1/3"><c>gl:evalMesh()</c></seemfa> are used together to efficiently generate and evaluate a series of evenly-spaced map domain values. <seemfa marker="gl#evalMesh1/3"><c>gl:evalMesh()</c></seemfa> steps through the integer domain of a one- or two-dimensional grid, whose range is the domain of the evaluation maps specified by <c>glMap1</c> and <c>glMap2</c>. </p> <p><url href="https://www.khronos.org/registry/OpenGL-Refpages/gl2.1/xhtml/glMapGrid.xml">External documentation.</url></p></desc> </func> @@ -2848,13 +2552,6 @@ <p><url href="https://www.khronos.org/registry/OpenGL-Refpages/gl2.1/xhtml/glMaterial.xml">External documentation.</url></p></desc> </func> <func> - <name name="matrixIndexubvARB" arity="1" clause_i="1" since=""/> - <name name="matrixIndexuivARB" arity="1" clause_i="1" since=""/> - <name name="matrixIndexusvARB" arity="1" clause_i="1" since=""/> - <fsummary/> - <desc><p>No documentation available.</p></desc> - </func> - <func> <name name="matrixMode" arity="1" clause_i="1" since=""/> <fsummary>specify which matrix is the current matrix</fsummary> <desc> @@ -2863,17 +2560,8 @@ <p><url href="https://www.khronos.org/registry/OpenGL-Refpages/gl2.1/xhtml/glMatrixMode.xml">External documentation.</url></p></desc> </func> <func> - <name name="maxShaderCompilerThreadsARB" arity="1" clause_i="1" since=""/> - <fsummary/> - <desc><p>No documentation available.</p></desc> - </func> - <func> - <name name="maxShaderCompilerThreadsKHR" arity="1" clause_i="1" since=""/> - <fsummary/> - <desc><p>No documentation available.</p></desc> - </func> - <func> <name name="memoryBarrier" arity="1" clause_i="1" since=""/> + <name name="memoryBarrierByRegion" arity="1" clause_i="1" since=""/> <fsummary>defines a barrier ordering memory transactions</fsummary> <desc> <p><seemfa marker="gl#memoryBarrier/1"><c>gl:memoryBarrier/1</c></seemfa> defines a barrier ordering the memory transactions issued prior to the command relative to those issued after the barrier. For the purposes of this ordering, memory transactions performed by shaders are considered to be issued by the rendering command that triggered the execution of the shader. <c>Barriers</c> is a bitfield indicating the set of operations that are synchronized with shader stores; the bits used in <c>Barriers</c> are as follows: </p> @@ -2881,11 +2569,6 @@ <p><url href="https://www.khronos.org/registry/OpenGL-Refpages/gl4/html/glMemoryBarrier.xhtml">External documentation.</url></p></desc> </func> <func> - <name name="memoryBarrierByRegion" arity="1" clause_i="1" since=""/> - <fsummary/> - <desc><p>No documentation available.</p></desc> - </func> - <func> <name name="minSampleShading" arity="1" clause_i="1" since=""/> <fsummary>specifies minimum rate at which sample shading takes place</fsummary> <desc> @@ -2920,12 +2603,6 @@ <p><url href="https://www.khronos.org/registry/OpenGL-Refpages/gl2.1/xhtml/glMultTransposeMatrix.xml">External documentation.</url></p></desc> </func> <func> - <name name="multTransposeMatrixdARB" arity="1" clause_i="1" since=""/> - <name name="multTransposeMatrixfARB" arity="1" clause_i="1" since=""/> - <fsummary/> - <desc><p>No documentation available.</p></desc> - </func> - <func> <name name="multiDrawArrays" arity="3" clause_i="1" since=""/> <fsummary>render multiple sets of primitives from array data</fsummary> <desc> @@ -2986,10 +2663,11 @@ <p><url href="https://www.khronos.org/registry/OpenGL-Refpages/gl2.1/xhtml/glMultiTexCoord.xml">External documentation.</url></p></desc> </func> <func> + <name name="endList" arity="0" clause_i="1" since=""/> <name name="newList" arity="2" clause_i="1" since=""/> <fsummary>create or replace a display list</fsummary> <desc> - <p> Display lists are groups of GL commands that have been stored for subsequent execution. Display lists are created with <seemfa marker="gl#newList/2"><c>gl:newList/2</c></seemfa>. All subsequent commands are placed in the display list, in the order issued, until <seemfa marker="gl#endList/0"><c>gl:endList/0</c></seemfa> is called. </p> + <p> Display lists are groups of GL commands that have been stored for subsequent execution. Display lists are created with <seemfa marker="gl#newList/2"><c>gl:newList/2</c></seemfa>. All subsequent commands are placed in the display list, in the order issued, until <seemfa marker="gl#newList/2"><c>gl:endList/0</c></seemfa> is called. </p> <p><url href="https://www.khronos.org/registry/OpenGL-Refpages/gl2.1/xhtml/glNewList.xml">External documentation.</url></p></desc> </func> @@ -3065,7 +2743,7 @@ <name name="pixelMapusv" arity="3" clause_i="1" since=""/> <fsummary>set up pixel transfer maps</fsummary> <desc> - <p><seemfa marker="gl#pixelMapfv/3"><c>gl:pixelMap()</c></seemfa> sets up translation tables, or <c>maps</c>, used by <seemfa marker="gl#copyPixels/5"><c>gl:copyPixels/5</c></seemfa>, <seemfa marker="gl#copyTexImage1D/7"><c>gl:copyTexImage1D/7</c></seemfa>, <seemfa marker="gl#copyTexImage2D/8"><c>gl:copyTexImage2D/8</c></seemfa>, <seemfa marker="gl#copyTexSubImage1D/6"><c>gl:copyTexSubImage1D/6</c></seemfa>, <seemfa marker="gl#copyTexSubImage2D/8"><c>gl:copyTexSubImage2D/8</c></seemfa>, <seemfa marker="gl#copyTexSubImage3D/9"><c>gl:copyTexSubImage3D/9</c></seemfa>, <seemfa marker="gl#drawPixels/5"><c>gl:drawPixels/5</c></seemfa>, <seemfa marker="gl#readPixels/7"><c>gl:readPixels/7</c></seemfa>, <seemfa marker="gl#texImage1D/8"><c>gl:texImage1D/8</c></seemfa>, <seemfa marker="gl#texImage2D/9"><c>gl:texImage2D/9</c></seemfa>, <seemfa marker="gl#texImage3D/10"><c>gl:texImage3D/10</c></seemfa>, <seemfa marker="gl#texSubImage1D/7"><c>gl:texSubImage1D/7</c></seemfa>, <seemfa marker="gl#texSubImage1D/7"><c>gl:texSubImage2D/9</c></seemfa>, and <seemfa marker="gl#texSubImage1D/7"><c>gl:texSubImage3D/11</c></seemfa>. Additionally, if the ARB_imaging subset is supported, the routines <seemfa marker="gl#colorTable/6"><c>gl:colorTable/6</c></seemfa>, <seemfa marker="gl#colorSubTable/6"><c>gl:colorSubTable/6</c></seemfa>, <seemfa marker="gl#convolutionFilter1D/6"><c>gl:convolutionFilter1D/6</c></seemfa>, <seemfa marker="gl#convolutionFilter2D/7"><c>gl:convolutionFilter2D/7</c></seemfa>, <seemfa marker="gl#histogram/4"><c>gl:histogram/4</c></seemfa>, <seemfa marker="gl#minmax/3"><c>gl:minmax/3</c></seemfa>, and <seemfa marker="gl#separableFilter2D/8"><c>gl:separableFilter2D/8</c></seemfa>. Use of these maps is described completely in the <seemfa marker="gl#pixelTransferf/2"><c>gl:pixelTransfer()</c></seemfa> reference page, and partly in the reference pages for the pixel and texture image commands. Only the specification of the maps is described in this reference page. </p> + <p><seemfa marker="gl#pixelMapfv/3"><c>gl:pixelMap()</c></seemfa> sets up translation tables, or <c>maps</c>, used by <seemfa marker="gl#copyPixels/5"><c>gl:copyPixels/5</c></seemfa>, <seemfa marker="gl#copyTexImage1D/7"><c>gl:copyTexImage1D/7</c></seemfa>, <seemfa marker="gl#copyTexImage2D/8"><c>gl:copyTexImage2D/8</c></seemfa>, <seemfa marker="gl#copyTexSubImage1D/6"><c>gl:copyTexSubImage1D/6</c></seemfa>, <seemfa marker="gl#copyTexSubImage2D/8"><c>gl:copyTexSubImage2D/8</c></seemfa>, <seemfa marker="gl#copyTexSubImage3D/9"><c>gl:copyTexSubImage3D/9</c></seemfa>, <seemfa marker="gl#drawPixels/5"><c>gl:drawPixels/5</c></seemfa>, <seemfa marker="gl#readPixels/7"><c>gl:readPixels/7</c></seemfa>, <seemfa marker="gl#texImage1D/8"><c>gl:texImage1D/8</c></seemfa>, <seemfa marker="gl#texImage2D/9"><c>gl:texImage2D/9</c></seemfa>, <seemfa marker="gl#texImage3D/10"><c>gl:texImage3D/10</c></seemfa>, <seemfa marker="gl#texSubImage1D/7"><c>gl:texSubImage1D/7</c></seemfa>, <seemfa marker="gl#texSubImage2D/9"><c>gl:texSubImage2D/9</c></seemfa>, and <seemfa marker="gl#texSubImage3D/11"><c>gl:texSubImage3D/11</c></seemfa>. Additionally, if the ARB_imaging subset is supported, the routines <seemfa marker="gl#colorTable/6"><c>gl:colorTable/6</c></seemfa>, <seemfa marker="gl#colorSubTable/6"><c>gl:colorSubTable/6</c></seemfa>, <seemfa marker="gl#convolutionFilter1D/6"><c>gl:convolutionFilter1D/6</c></seemfa>, <seemfa marker="gl#convolutionFilter2D/7"><c>gl:convolutionFilter2D/7</c></seemfa>, <seemfa marker="gl#histogram/4"><c>gl:histogram/4</c></seemfa>, <seemfa marker="gl#minmax/3"><c>gl:minmax/3</c></seemfa>, and <seemfa marker="gl#separableFilter2D/8"><c>gl:separableFilter2D/8</c></seemfa>. Use of these maps is described completely in the <seemfa marker="gl#pixelTransferf/2"><c>gl:pixelTransfer()</c></seemfa> reference page, and partly in the reference pages for the pixel and texture image commands. Only the specification of the maps is described in this reference page. </p> <p><url href="https://www.khronos.org/registry/OpenGL-Refpages/gl2.1/xhtml/glPixelMap.xml">External documentation.</url></p></desc> </func> @@ -3074,7 +2752,7 @@ <name name="pixelStorei" arity="2" clause_i="1" since=""/> <fsummary>set pixel storage modes</fsummary> <desc> - <p><seemfa marker="gl#pixelStoref/2"><c>gl:pixelStore()</c></seemfa> sets pixel storage modes that affect the operation of subsequent <seemfa marker="gl#readPixels/7"><c>gl:readPixels/7</c></seemfa> as well as the unpacking of texture patterns (see <seemfa marker="gl#texImage1D/8"><c>gl:texImage1D/8</c></seemfa>, <seemfa marker="gl#texImage2D/9"><c>gl:texImage2D/9</c></seemfa>, <seemfa marker="gl#texImage3D/10"><c>gl:texImage3D/10</c></seemfa>, <seemfa marker="gl#texSubImage1D/7"><c>gl:texSubImage1D/7</c></seemfa>, <seemfa marker="gl#texSubImage1D/7"><c>gl:texSubImage2D/9</c></seemfa>, <seemfa marker="gl#texSubImage1D/7"><c>gl:texSubImage3D/11</c></seemfa>), <seemfa marker="gl#compressedTexImage1D/7"><c>gl:compressedTexImage1D/7</c></seemfa>, <seemfa marker="gl#compressedTexImage2D/8"><c>gl:compressedTexImage2D/8</c></seemfa>, <seemfa marker="gl#compressedTexImage3D/9"><c>gl:compressedTexImage3D/9</c></seemfa>, <seemfa marker="gl#compressedTexSubImage1D/7"><c>gl:compressedTexSubImage1D/7</c></seemfa>, <seemfa marker="gl#compressedTexSubImage2D/9"><c>gl:compressedTexSubImage2D/9</c></seemfa> or <seemfa marker="gl#compressedTexSubImage1D/7"><c>gl:compressedTexSubImage1D/7</c></seemfa>. </p> + <p><seemfa marker="gl#pixelStoref/2"><c>gl:pixelStore()</c></seemfa> sets pixel storage modes that affect the operation of subsequent <seemfa marker="gl#readPixels/7"><c>gl:readPixels/7</c></seemfa> as well as the unpacking of texture patterns (see <seemfa marker="gl#texImage1D/8"><c>gl:texImage1D/8</c></seemfa>, <seemfa marker="gl#texImage2D/9"><c>gl:texImage2D/9</c></seemfa>, <seemfa marker="gl#texImage3D/10"><c>gl:texImage3D/10</c></seemfa>, <seemfa marker="gl#texSubImage1D/7"><c>gl:texSubImage1D/7</c></seemfa>, <seemfa marker="gl#texSubImage2D/9"><c>gl:texSubImage2D/9</c></seemfa>, <seemfa marker="gl#texSubImage3D/11"><c>gl:texSubImage3D/11</c></seemfa>), <seemfa marker="gl#compressedTexImage1D/7"><c>gl:compressedTexImage1D/7</c></seemfa>, <seemfa marker="gl#compressedTexImage2D/8"><c>gl:compressedTexImage2D/8</c></seemfa>, <seemfa marker="gl#compressedTexImage3D/9"><c>gl:compressedTexImage3D/9</c></seemfa>, <seemfa marker="gl#compressedTexSubImage1D/7"><c>gl:compressedTexSubImage1D/7</c></seemfa>, <seemfa marker="gl#compressedTexSubImage2D/9"><c>gl:compressedTexSubImage2D/9</c></seemfa> or <seemfa marker="gl#compressedTexSubImage1D/7"><c>gl:compressedTexSubImage1D/7</c></seemfa>. </p> <p><url href="https://www.khronos.org/registry/OpenGL-Refpages/gl4/html/glPixelStore.xhtml">External documentation.</url></p></desc> </func> @@ -3083,7 +2761,7 @@ <name name="pixelTransferi" arity="2" clause_i="1" since=""/> <fsummary>set pixel transfer modes</fsummary> <desc> - <p><seemfa marker="gl#pixelTransferf/2"><c>gl:pixelTransfer()</c></seemfa> sets pixel transfer modes that affect the operation of subsequent <seemfa marker="gl#copyPixels/5"><c>gl:copyPixels/5</c></seemfa>, <seemfa marker="gl#copyTexImage1D/7"><c>gl:copyTexImage1D/7</c></seemfa>, <seemfa marker="gl#copyTexImage2D/8"><c>gl:copyTexImage2D/8</c></seemfa>, <seemfa marker="gl#copyTexSubImage1D/6"><c>gl:copyTexSubImage1D/6</c></seemfa>, <seemfa marker="gl#copyTexSubImage2D/8"><c>gl:copyTexSubImage2D/8</c></seemfa>, <seemfa marker="gl#copyTexSubImage3D/9"><c>gl:copyTexSubImage3D/9</c></seemfa>, <seemfa marker="gl#drawPixels/5"><c>gl:drawPixels/5</c></seemfa>, <seemfa marker="gl#readPixels/7"><c>gl:readPixels/7</c></seemfa>, <seemfa marker="gl#texImage1D/8"><c>gl:texImage1D/8</c></seemfa>, <seemfa marker="gl#texImage2D/9"><c>gl:texImage2D/9</c></seemfa>, <seemfa marker="gl#texImage3D/10"><c>gl:texImage3D/10</c></seemfa>, <seemfa marker="gl#texSubImage1D/7"><c>gl:texSubImage1D/7</c></seemfa>, <seemfa marker="gl#texSubImage1D/7"><c>gl:texSubImage2D/9</c></seemfa>, and <seemfa marker="gl#texSubImage1D/7"><c>gl:texSubImage3D/11</c></seemfa> commands. Additionally, if the ARB_imaging subset is supported, the routines <seemfa marker="gl#colorTable/6"><c>gl:colorTable/6</c></seemfa>, <seemfa marker="gl#colorSubTable/6"><c>gl:colorSubTable/6</c></seemfa>, <seemfa marker="gl#convolutionFilter1D/6"><c>gl:convolutionFilter1D/6</c></seemfa>, <seemfa marker="gl#convolutionFilter2D/7"><c>gl:convolutionFilter2D/7</c></seemfa>, <seemfa marker="gl#histogram/4"><c>gl:histogram/4</c></seemfa>, <seemfa marker="gl#minmax/3"><c>gl:minmax/3</c></seemfa>, and <seemfa marker="gl#separableFilter2D/8"><c>gl:separableFilter2D/8</c></seemfa> are also affected. The algorithms that are specified by pixel transfer modes operate on pixels after they are read from the frame buffer (<seemfa marker="gl#copyPixels/5"><c>gl:copyPixels/5</c></seemfa><seemfa marker="gl#copyTexImage1D/7"><c>gl:copyTexImage1D/7</c></seemfa>, <seemfa marker="gl#copyTexImage2D/8"><c>gl:copyTexImage2D/8</c></seemfa>, <seemfa marker="gl#copyTexSubImage1D/6"><c>gl:copyTexSubImage1D/6</c></seemfa>, <seemfa marker="gl#copyTexSubImage2D/8"><c>gl:copyTexSubImage2D/8</c></seemfa>, <seemfa marker="gl#copyTexSubImage3D/9"><c>gl:copyTexSubImage3D/9</c></seemfa>, and <seemfa marker="gl#readPixels/7"><c>gl:readPixels/7</c></seemfa>), or unpacked from client memory (<seemfa marker="gl#drawPixels/5"><c>gl:drawPixels/5</c></seemfa>, <seemfa marker="gl#texImage1D/8"><c>gl:texImage1D/8</c></seemfa>, <seemfa marker="gl#texImage2D/9"><c>gl:texImage2D/9</c></seemfa>, <seemfa marker="gl#texImage3D/10"><c>gl:texImage3D/10</c></seemfa>, <seemfa marker="gl#texSubImage1D/7"><c>gl:texSubImage1D/7</c></seemfa>, <seemfa marker="gl#texSubImage1D/7"><c>gl:texSubImage2D/9</c></seemfa>, and <seemfa marker="gl#texSubImage1D/7"><c>gl:texSubImage3D/11</c></seemfa>). Pixel transfer operations happen in the same order, and in the same manner, regardless of the command that resulted in the pixel operation. Pixel storage modes (see <seemfa marker="gl#pixelStoref/2"><c>gl:pixelStore()</c></seemfa>) control the unpacking of pixels being read from client memory and the packing of pixels being written back into client memory. </p> + <p><seemfa marker="gl#pixelTransferf/2"><c>gl:pixelTransfer()</c></seemfa> sets pixel transfer modes that affect the operation of subsequent <seemfa marker="gl#copyPixels/5"><c>gl:copyPixels/5</c></seemfa>, <seemfa marker="gl#copyTexImage1D/7"><c>gl:copyTexImage1D/7</c></seemfa>, <seemfa marker="gl#copyTexImage2D/8"><c>gl:copyTexImage2D/8</c></seemfa>, <seemfa marker="gl#copyTexSubImage1D/6"><c>gl:copyTexSubImage1D/6</c></seemfa>, <seemfa marker="gl#copyTexSubImage2D/8"><c>gl:copyTexSubImage2D/8</c></seemfa>, <seemfa marker="gl#copyTexSubImage3D/9"><c>gl:copyTexSubImage3D/9</c></seemfa>, <seemfa marker="gl#drawPixels/5"><c>gl:drawPixels/5</c></seemfa>, <seemfa marker="gl#readPixels/7"><c>gl:readPixels/7</c></seemfa>, <seemfa marker="gl#texImage1D/8"><c>gl:texImage1D/8</c></seemfa>, <seemfa marker="gl#texImage2D/9"><c>gl:texImage2D/9</c></seemfa>, <seemfa marker="gl#texImage3D/10"><c>gl:texImage3D/10</c></seemfa>, <seemfa marker="gl#texSubImage1D/7"><c>gl:texSubImage1D/7</c></seemfa>, <seemfa marker="gl#texSubImage2D/9"><c>gl:texSubImage2D/9</c></seemfa>, and <seemfa marker="gl#texSubImage3D/11"><c>gl:texSubImage3D/11</c></seemfa> commands. Additionally, if the ARB_imaging subset is supported, the routines <seemfa marker="gl#colorTable/6"><c>gl:colorTable/6</c></seemfa>, <seemfa marker="gl#colorSubTable/6"><c>gl:colorSubTable/6</c></seemfa>, <seemfa marker="gl#convolutionFilter1D/6"><c>gl:convolutionFilter1D/6</c></seemfa>, <seemfa marker="gl#convolutionFilter2D/7"><c>gl:convolutionFilter2D/7</c></seemfa>, <seemfa marker="gl#histogram/4"><c>gl:histogram/4</c></seemfa>, <seemfa marker="gl#minmax/3"><c>gl:minmax/3</c></seemfa>, and <seemfa marker="gl#separableFilter2D/8"><c>gl:separableFilter2D/8</c></seemfa> are also affected. The algorithms that are specified by pixel transfer modes operate on pixels after they are read from the frame buffer (<seemfa marker="gl#copyPixels/5"><c>gl:copyPixels/5</c></seemfa><seemfa marker="gl#copyTexImage1D/7"><c>gl:copyTexImage1D/7</c></seemfa>, <seemfa marker="gl#copyTexImage2D/8"><c>gl:copyTexImage2D/8</c></seemfa>, <seemfa marker="gl#copyTexSubImage1D/6"><c>gl:copyTexSubImage1D/6</c></seemfa>, <seemfa marker="gl#copyTexSubImage2D/8"><c>gl:copyTexSubImage2D/8</c></seemfa>, <seemfa marker="gl#copyTexSubImage3D/9"><c>gl:copyTexSubImage3D/9</c></seemfa>, and <seemfa marker="gl#readPixels/7"><c>gl:readPixels/7</c></seemfa>), or unpacked from client memory (<seemfa marker="gl#drawPixels/5"><c>gl:drawPixels/5</c></seemfa>, <seemfa marker="gl#texImage1D/8"><c>gl:texImage1D/8</c></seemfa>, <seemfa marker="gl#texImage2D/9"><c>gl:texImage2D/9</c></seemfa>, <seemfa marker="gl#texImage3D/10"><c>gl:texImage3D/10</c></seemfa>, <seemfa marker="gl#texSubImage1D/7"><c>gl:texSubImage1D/7</c></seemfa>, <seemfa marker="gl#texSubImage2D/9"><c>gl:texSubImage2D/9</c></seemfa>, and <seemfa marker="gl#texSubImage3D/11"><c>gl:texSubImage3D/11</c></seemfa>). Pixel transfer operations happen in the same order, and in the same manner, regardless of the command that resulted in the pixel operation. Pixel storage modes (see <seemfa marker="gl#pixelStoref/2"><c>gl:pixelStore()</c></seemfa>) control the unpacking of pixels being read from client memory and the packing of pixels being written back into client memory. </p> <p><url href="https://www.khronos.org/registry/OpenGL-Refpages/gl2.1/xhtml/glPixelTransfer.xml">External documentation.</url></p></desc> </func> @@ -3144,11 +2822,6 @@ <p><url href="https://www.khronos.org/registry/OpenGL-Refpages/gl2.1/xhtml/glPolygonStipple.xml">External documentation.</url></p></desc> </func> <func> - <name name="primitiveBoundingBoxARB" arity="8" clause_i="1" since=""/> - <fsummary/> - <desc><p>No documentation available.</p></desc> - </func> - <func> <name name="primitiveRestartIndex" arity="1" clause_i="1" since=""/> <fsummary>specify the primitive restart index</fsummary> <desc> @@ -3173,22 +2846,6 @@ <p><url href="https://www.khronos.org/registry/OpenGL-Refpages/gl4/html/glProgramBinary.xhtml">External documentation.</url></p></desc> </func> <func> - <name name="programEnvParameter4dARB" arity="6" clause_i="1" since=""/> - <name name="programEnvParameter4dvARB" arity="3" clause_i="1" since=""/> - <name name="programEnvParameter4fARB" arity="6" clause_i="1" since=""/> - <name name="programEnvParameter4fvARB" arity="3" clause_i="1" since=""/> - <fsummary/> - <desc><p>No documentation available.</p></desc> - </func> - <func> - <name name="programLocalParameter4dARB" arity="6" clause_i="1" since=""/> - <name name="programLocalParameter4dvARB" arity="3" clause_i="1" since=""/> - <name name="programLocalParameter4fARB" arity="6" clause_i="1" since=""/> - <name name="programLocalParameter4fvARB" arity="3" clause_i="1" since=""/> - <fsummary/> - <desc><p>No documentation available.</p></desc> - </func> - <func> <name name="programParameteri" arity="3" clause_i="1" since=""/> <fsummary>specify a parameter for a program object</fsummary> <desc> @@ -3197,11 +2854,6 @@ <p><url href="https://www.khronos.org/registry/OpenGL-Refpages/gl4/html/glProgramParameter.xhtml">External documentation.</url></p></desc> </func> <func> - <name name="programStringARB" arity="3" clause_i="1" since=""/> - <fsummary/> - <desc><p>No documentation available.</p></desc> - </func> - <func> <name name="programUniform1d" arity="3" clause_i="1" since=""/> <name name="programUniform1dv" arity="3" clause_i="1" since=""/> <name name="programUniform1f" arity="3" clause_i="1" since=""/> @@ -3259,91 +2911,6 @@ <p><url href="https://www.khronos.org/registry/OpenGL-Refpages/gl4/html/glProgramUniform.xhtml">External documentation.</url></p></desc> </func> <func> - <name name="programUniform1i64vARB" arity="3" clause_i="1" since=""/> - <fsummary/> - <desc><p>No documentation available.</p></desc> - </func> - <func> - <name name="programUniform1i64ARB" arity="3" clause_i="1" since=""/> - <fsummary/> - <desc><p>No documentation available.</p></desc> - </func> - <func> - <name name="programUniform1ui64vARB" arity="3" clause_i="1" since=""/> - <fsummary/> - <desc><p>No documentation available.</p></desc> - </func> - <func> - <name name="programUniform1ui64ARB" arity="3" clause_i="1" since=""/> - <fsummary/> - <desc><p>No documentation available.</p></desc> - </func> - <func> - <name name="programUniform2i64vARB" arity="3" clause_i="1" since=""/> - <fsummary/> - <desc><p>No documentation available.</p></desc> - </func> - <func> - <name name="programUniform2i64ARB" arity="4" clause_i="1" since=""/> - <fsummary/> - <desc><p>No documentation available.</p></desc> - </func> - <func> - <name name="programUniform2ui64vARB" arity="3" clause_i="1" since=""/> - <fsummary/> - <desc><p>No documentation available.</p></desc> - </func> - <func> - <name name="programUniform2ui64ARB" arity="4" clause_i="1" since=""/> - <fsummary/> - <desc><p>No documentation available.</p></desc> - </func> - <func> - <name name="programUniform3i64vARB" arity="3" clause_i="1" since=""/> - <fsummary/> - <desc><p>No documentation available.</p></desc> - </func> - <func> - <name name="programUniform3i64ARB" arity="5" clause_i="1" since=""/> - <fsummary/> - <desc><p>No documentation available.</p></desc> - </func> - <func> - <name name="programUniform3ui64vARB" arity="3" clause_i="1" since=""/> - <fsummary/> - <desc><p>No documentation available.</p></desc> - </func> - <func> - <name name="programUniform3ui64ARB" arity="5" clause_i="1" since=""/> - <fsummary/> - <desc><p>No documentation available.</p></desc> - </func> - <func> - <name name="programUniform4i64vARB" arity="3" clause_i="1" since=""/> - <fsummary/> - <desc><p>No documentation available.</p></desc> - </func> - <func> - <name name="programUniform4i64ARB" arity="6" clause_i="1" since=""/> - <fsummary/> - <desc><p>No documentation available.</p></desc> - </func> - <func> - <name name="programUniform4ui64vARB" arity="3" clause_i="1" since=""/> - <fsummary/> - <desc><p>No documentation available.</p></desc> - </func> - <func> - <name name="programUniform4ui64ARB" arity="6" clause_i="1" since=""/> - <fsummary/> - <desc><p>No documentation available.</p></desc> - </func> - <func> - <name name="programUniformHandleui64ARB" arity="3" clause_i="1" since=""/> - <fsummary/> - <desc><p>No documentation available.</p></desc> - </func> - <func> <name name="provokingVertex" arity="1" clause_i="1" since=""/> <fsummary>specifiy the vertex to be used as the source of data for flat shaded varyings</fsummary> <desc> @@ -3540,11 +3107,6 @@ <p><url href="https://www.khronos.org/registry/OpenGL-Refpages/gl4/html/glSampleCoverage.xhtml">External documentation.</url></p></desc> </func> <func> - <name name="sampleCoverageARB" arity="2" clause_i="1" since=""/> - <fsummary/> - <desc><p>No documentation available.</p></desc> - </func> - <func> <name name="sampleMaski" arity="2" clause_i="1" since=""/> <fsummary>set the value of a sub-word of the sample mask</fsummary> <desc> @@ -3554,6 +3116,7 @@ </func> <func> <name name="samplerParameterIiv" arity="3" clause_i="1" since=""/> + <name name="samplerParameterIuiv" arity="3" clause_i="1" since=""/> <name name="samplerParameterf" arity="3" clause_i="1" since=""/> <name name="samplerParameterfv" arity="3" clause_i="1" since=""/> <name name="samplerParameteri" arity="3" clause_i="1" since=""/> @@ -3565,11 +3128,6 @@ <p><url href="https://www.khronos.org/registry/OpenGL-Refpages/gl4/html/glSamplerParameter.xhtml">External documentation.</url></p></desc> </func> <func> - <name name="samplerParameterIuiv" arity="3" clause_i="1" since=""/> - <fsummary/> - <desc><p>No documentation available.</p></desc> - </func> - <func> <name name="scaled" arity="3" clause_i="1" since=""/> <name name="scalef" arity="3" clause_i="1" since=""/> <fsummary>multiply the current matrix by a general scaling matrix</fsummary> @@ -3588,14 +3146,20 @@ </func> <func> <name name="scissorArrayv" arity="2" clause_i="1" since=""/> - <fsummary/> - <desc><p>No documentation available.</p></desc> + <fsummary>define the scissor box for multiple viewports</fsummary> + <desc> + <p><seemfa marker="gl#scissorArrayv/2"><c>gl:scissorArrayv/2</c></seemfa> defines rectangles, called scissor boxes, in window coordinates for each viewport. <c>First</c> specifies the index of the first scissor box to modify and <c>Count</c> specifies the number of scissor boxes to modify. <c>First</c> must be less than the value of <c>?GL_MAX_VIEWPORTS</c>, and <c>First</c> + <c>Count</c> must be less than or equal to the value of <c>?GL_MAX_VIEWPORTS</c>. <c>V</c> specifies the address of an array containing integers specifying the lower left corner of the scissor boxes, and the width and height of the scissor boxes, in that order. </p> + + <p><url href="https://www.khronos.org/registry/OpenGL-Refpages/gl4/html/glScissorArray.xhtml">External documentation.</url></p></desc> </func> <func> <name name="scissorIndexed" arity="5" clause_i="1" since=""/> <name name="scissorIndexedv" arity="2" clause_i="1" since=""/> - <fsummary/> - <desc><p>No documentation available.</p></desc> + <fsummary>define the scissor box for a specific viewport</fsummary> + <desc> + <p><seemfa marker="gl#scissorIndexed/5"><c>gl:scissorIndexed/5</c></seemfa> defines the scissor box for a specified viewport. <c>Index</c> specifies the index of scissor box to modify. <c>Index</c> must be less than the value of <c>?GL_MAX_VIEWPORTS</c>. For <seemfa marker="gl#scissorIndexed/5"><c>gl:scissorIndexed/5</c></seemfa>, <c>Left</c>, <c>Bottom</c>, <c>Width</c> and <c>Height</c> specify the left, bottom, width and height of the scissor box, in pixels, respectively. For <seemfa marker="gl#scissorIndexed/5"><c>gl:scissorIndexedv/2</c></seemfa>, <c>V</c> specifies the address of an array containing integers specifying the lower left corner of the scissor box, and the width and height of the scissor box, in that order. </p> + + <p><url href="https://www.khronos.org/registry/OpenGL-Refpages/gl4/html/glScissorIndexed.xhtml">External documentation.</url></p></desc> </func> <func> <name name="secondaryColor3b" arity="3" clause_i="1" since=""/> @@ -3669,11 +3233,6 @@ <p><url href="https://www.khronos.org/registry/OpenGL-Refpages/gl4/html/glShaderSource.xhtml">External documentation.</url></p></desc> </func> <func> - <name name="shaderSourceARB" arity="2" clause_i="1" since=""/> - <fsummary/> - <desc><p>No documentation available.</p></desc> - </func> - <func> <name name="shaderStorageBlockBinding" arity="3" clause_i="1" since=""/> <fsummary>change an active shader storage block binding</fsummary> <desc> @@ -3731,17 +3290,19 @@ </func> <func> <name name="texBuffer" arity="3" clause_i="1" since=""/> + <name name="textureBuffer" arity="3" clause_i="1" since=""/> <fsummary>attach a buffer object's data store to a buffer texture object</fsummary> <desc> - <p><seemfa marker="gl#texBuffer/3"><c>gl:texBuffer/3</c></seemfa> and <seemfa marker="gl#textureBuffer/3"><c>gl:textureBuffer/3</c></seemfa> attaches the data store of a specified buffer object to a specified texture object, and specify the storage format for the texture image found in the buffer object. The texture object must be a buffer texture. </p> + <p><seemfa marker="gl#texBuffer/3"><c>gl:texBuffer/3</c></seemfa> and <seemfa marker="gl#texBuffer/3"><c>gl:textureBuffer/3</c></seemfa> attaches the data store of a specified buffer object to a specified texture object, and specify the storage format for the texture image found in the buffer object. The texture object must be a buffer texture. </p> <p><url href="https://www.khronos.org/registry/OpenGL-Refpages/gl4/html/glTexBuffer.xhtml">External documentation.</url></p></desc> </func> <func> <name name="texBufferRange" arity="5" clause_i="1" since=""/> + <name name="textureBufferRange" arity="5" clause_i="1" since=""/> <fsummary>attach a range of a buffer object's data store to a buffer texture object</fsummary> <desc> - <p><seemfa marker="gl#texBufferRange/5"><c>gl:texBufferRange/5</c></seemfa> and <seemfa marker="gl#textureBufferRange/5"><c>gl:textureBufferRange/5</c></seemfa> attach a range of the data store of a specified buffer object to a specified texture object, and specify the storage format for the texture image found in the buffer object. The texture object must be a buffer texture. </p> + <p><seemfa marker="gl#texBufferRange/5"><c>gl:texBufferRange/5</c></seemfa> and <seemfa marker="gl#texBufferRange/5"><c>gl:textureBufferRange/5</c></seemfa> attach a range of the data store of a specified buffer object to a specified texture object, and specify the storage format for the texture image found in the buffer object. The texture object must be a buffer texture. </p> <p><url href="https://www.khronos.org/registry/OpenGL-Refpages/gl4/html/glTexBufferRange.xhtml">External documentation.</url></p></desc> </func> @@ -3793,7 +3354,9 @@ <p><url href="https://www.khronos.org/registry/OpenGL-Refpages/gl2.1/xhtml/glTexCoordPointer.xml">External documentation.</url></p></desc> </func> <func> + <name name="texEnvf" arity="3" clause_i="1" since=""/> <name name="texEnvfv" arity="3" clause_i="1" since=""/> + <name name="texEnvi" arity="3" clause_i="1" since=""/> <name name="texEnviv" arity="3" clause_i="1" since=""/> <fsummary>set texture environment parameters</fsummary> <desc> @@ -3802,16 +3365,6 @@ <p><url href="https://www.khronos.org/registry/OpenGL-Refpages/gl2.1/xhtml/glTexEnv.xml">External documentation.</url></p></desc> </func> <func> - <name name="texEnvf" arity="3" clause_i="1" since=""/> - <fsummary/> - <desc><p>No documentation available.</p></desc> - </func> - <func> - <name name="texEnvi" arity="3" clause_i="1" since=""/> - <fsummary/> - <desc><p>No documentation available.</p></desc> - </func> - <func> <name name="texGend" arity="3" clause_i="1" since=""/> <name name="texGendv" arity="3" clause_i="1" since=""/> <name name="texGenf" arity="3" clause_i="1" since=""/> @@ -3865,32 +3418,23 @@ <p><url href="https://www.khronos.org/registry/OpenGL-Refpages/gl4/html/glTexImage3DMultisample.xhtml">External documentation.</url></p></desc> </func> <func> - <name name="texPageCommitmentARB" arity="9" clause_i="1" since=""/> - <fsummary/> - <desc><p>No documentation available.</p></desc> - </func> - <func> <name name="texParameterIiv" arity="3" clause_i="1" since=""/> + <name name="texParameterIuiv" arity="3" clause_i="1" since=""/> <name name="texParameterf" arity="3" clause_i="1" since=""/> <name name="texParameterfv" arity="3" clause_i="1" since=""/> <name name="texParameteri" arity="3" clause_i="1" since=""/> <name name="texParameteriv" arity="3" clause_i="1" since=""/> <fsummary>set texture parameters</fsummary> <desc> - <p><seemfa marker="gl#texParameterf/3"><c>gl:texParameter()</c></seemfa> and <c>glTextureParameter</c> assign the value or values in <c>Params</c> to the texture parameter specified as <c>Pname</c>. For <seemfa marker="gl#texParameterf/3"><c>gl:texParameter()</c></seemfa>, <c>Target</c> defines the target texture, either <c>?GL_TEXTURE_1D</c>, <c>?GL_TEXTURE_1D_ARRAY</c>, <c>?GL_TEXTURE_2D</c>, <c>?GL_TEXTURE_2D_ARRAY</c>, <c>?GL_TEXTURE_2D_MULTISAMPLE</c>, <c>?GL_TEXTURE_2D_MULTISAMPLE_ARRAY</c>, <c>?GL_TEXTURE_3D</c>, <c>?GL_TEXTURE_CUBE_MAP</c>, <c>?GL_TEXTURE_CUBE_MAP_ARRAY</c>, or <c>?GL_TEXTURE_RECTANGLE</c>. The following symbols are accepted in <c>Pname</c>:</p> + <p><seemfa marker="gl#texParameterf/3"><c>gl:texParameter()</c></seemfa> and <seemfa marker="gl#texParameterf/3"><c>gl:textureParameter()</c></seemfa> assign the value or values in <c>Params</c> to the texture parameter specified as <c>Pname</c>. For <seemfa marker="gl#texParameterf/3"><c>gl:texParameter()</c></seemfa>, <c>Target</c> defines the target texture, either <c>?GL_TEXTURE_1D</c>, <c>?GL_TEXTURE_1D_ARRAY</c>, <c>?GL_TEXTURE_2D</c>, <c>?GL_TEXTURE_2D_ARRAY</c>, <c>?GL_TEXTURE_2D_MULTISAMPLE</c>, <c>?GL_TEXTURE_2D_MULTISAMPLE_ARRAY</c>, <c>?GL_TEXTURE_3D</c>, <c>?GL_TEXTURE_CUBE_MAP</c>, <c>?GL_TEXTURE_CUBE_MAP_ARRAY</c>, or <c>?GL_TEXTURE_RECTANGLE</c>. The following symbols are accepted in <c>Pname</c>:</p> <p><url href="https://www.khronos.org/registry/OpenGL-Refpages/gl4/html/glTexParameter.xhtml">External documentation.</url></p></desc> </func> <func> - <name name="texParameterIuiv" arity="3" clause_i="1" since=""/> - <fsummary/> - <desc><p>No documentation available.</p></desc> - </func> - <func> <name name="texStorage1D" arity="4" clause_i="1" since=""/> <fsummary>simultaneously specify storage for all levels of a one-dimensional texture</fsummary> <desc> - <p><seemfa marker="gl#texStorage1D/4"><c>gl:texStorage1D/4</c></seemfa> and <c>glTextureStorage1D</c> specify the storage requirements for all levels of a one-dimensional texture simultaneously. Once a texture is specified with this command, the format and dimensions of all levels become immutable unless it is a proxy texture. The contents of the image may still be modified, however, its storage requirements may not change. Such a texture is referred to as an <c>immutable-format</c> texture. </p> + <p><seemfa marker="gl#texStorage1D/4"><c>gl:texStorage1D/4</c></seemfa> and <seemfa marker="gl#texStorage1D/4"><c>gl:textureStorage1D()</c></seemfa> specify the storage requirements for all levels of a one-dimensional texture simultaneously. Once a texture is specified with this command, the format and dimensions of all levels become immutable unless it is a proxy texture. The contents of the image may still be modified, however, its storage requirements may not change. Such a texture is referred to as an <c>immutable-format</c> texture. </p> <p><url href="https://www.khronos.org/registry/OpenGL-Refpages/gl4/html/glTexStorage1D.xhtml">External documentation.</url></p></desc> </func> @@ -3898,7 +3442,7 @@ <name name="texStorage2D" arity="5" clause_i="1" since=""/> <fsummary>simultaneously specify storage for all levels of a two-dimensional or one-dimensional array texture</fsummary> <desc> - <p><seemfa marker="gl#texStorage2D/5"><c>gl:texStorage2D/5</c></seemfa> and <c>glTextureStorage2D</c> specify the storage requirements for all levels of a two-dimensional texture or one-dimensional texture array simultaneously. Once a texture is specified with this command, the format and dimensions of all levels become immutable unless it is a proxy texture. The contents of the image may still be modified, however, its storage requirements may not change. Such a texture is referred to as an <c>immutable-format</c> texture. </p> + <p><seemfa marker="gl#texStorage2D/5"><c>gl:texStorage2D/5</c></seemfa> and <seemfa marker="gl#texStorage2D/5"><c>gl:textureStorage2D()</c></seemfa> specify the storage requirements for all levels of a two-dimensional texture or one-dimensional texture array simultaneously. Once a texture is specified with this command, the format and dimensions of all levels become immutable unless it is a proxy texture. The contents of the image may still be modified, however, its storage requirements may not change. Such a texture is referred to as an <c>immutable-format</c> texture. </p> <p><url href="https://www.khronos.org/registry/OpenGL-Refpages/gl4/html/glTexStorage2D.xhtml">External documentation.</url></p></desc> </func> @@ -3906,7 +3450,7 @@ <name name="texStorage2DMultisample" arity="6" clause_i="1" since=""/> <fsummary>specify storage for a two-dimensional multisample texture</fsummary> <desc> - <p><seemfa marker="gl#texStorage2DMultisample/6"><c>gl:texStorage2DMultisample/6</c></seemfa> and <c>glTextureStorage2DMultisample</c> specify the storage requirements for a two-dimensional multisample texture. Once a texture is specified with this command, its format and dimensions become immutable unless it is a proxy texture. The contents of the image may still be modified, however, its storage requirements may not change. Such a texture is referred to as an <c>immutable-format</c> texture. </p> + <p><seemfa marker="gl#texStorage2DMultisample/6"><c>gl:texStorage2DMultisample/6</c></seemfa> and <seemfa marker="gl#texStorage2DMultisample/6"><c>gl:textureStorage2DMultisample()</c></seemfa> specify the storage requirements for a two-dimensional multisample texture. Once a texture is specified with this command, its format and dimensions become immutable unless it is a proxy texture. The contents of the image may still be modified, however, its storage requirements may not change. Such a texture is referred to as an <c>immutable-format</c> texture. </p> <p><url href="https://www.khronos.org/registry/OpenGL-Refpages/gl4/html/glTexStorage2DMultisample.xhtml">External documentation.</url></p></desc> </func> @@ -3914,7 +3458,7 @@ <name name="texStorage3D" arity="6" clause_i="1" since=""/> <fsummary>simultaneously specify storage for all levels of a three-dimensional, two-dimensional array or cube-map array texture</fsummary> <desc> - <p><seemfa marker="gl#texStorage3D/6"><c>gl:texStorage3D/6</c></seemfa> and <c>glTextureStorage3D</c> specify the storage requirements for all levels of a three-dimensional, two-dimensional array or cube-map array texture simultaneously. Once a texture is specified with this command, the format and dimensions of all levels become immutable unless it is a proxy texture. The contents of the image may still be modified, however, its storage requirements may not change. Such a texture is referred to as an <c>immutable-format</c> texture. </p> + <p><seemfa marker="gl#texStorage3D/6"><c>gl:texStorage3D/6</c></seemfa> and <seemfa marker="gl#texStorage3D/6"><c>gl:textureStorage3D()</c></seemfa> specify the storage requirements for all levels of a three-dimensional, two-dimensional array or cube-map array texture simultaneously. Once a texture is specified with this command, the format and dimensions of all levels become immutable unless it is a proxy texture. The contents of the image may still be modified, however, its storage requirements may not change. Such a texture is referred to as an <c>immutable-format</c> texture. </p> <p><url href="https://www.khronos.org/registry/OpenGL-Refpages/gl4/html/glTexStorage3D.xhtml">External documentation.</url></p></desc> </func> @@ -3922,16 +3466,33 @@ <name name="texStorage3DMultisample" arity="7" clause_i="1" since=""/> <fsummary>specify storage for a two-dimensional multisample array texture</fsummary> <desc> - <p><seemfa marker="gl#texStorage3DMultisample/7"><c>gl:texStorage3DMultisample/7</c></seemfa> and <c>glTextureStorage3DMultisample</c> specify the storage requirements for a two-dimensional multisample array texture. Once a texture is specified with this command, its format and dimensions become immutable unless it is a proxy texture. The contents of the image may still be modified, however, its storage requirements may not change. Such a texture is referred to as an <c>immutable-format</c> texture. </p> + <p><seemfa marker="gl#texStorage3DMultisample/7"><c>gl:texStorage3DMultisample/7</c></seemfa> and <seemfa marker="gl#texStorage3DMultisample/7"><c>gl:textureStorage3DMultisample()</c></seemfa> specify the storage requirements for a two-dimensional multisample array texture. Once a texture is specified with this command, its format and dimensions become immutable unless it is a proxy texture. The contents of the image may still be modified, however, its storage requirements may not change. Such a texture is referred to as an <c>immutable-format</c> texture. </p> <p><url href="https://www.khronos.org/registry/OpenGL-Refpages/gl4/html/glTexStorage3DMultisample.xhtml">External documentation.</url></p></desc> </func> <func> <name name="texSubImage1D" arity="7" clause_i="1" since=""/> + <fsummary>specify a one-dimensional texture subimage</fsummary> + <desc> + <p> Texturing maps a portion of a specified texture image onto each graphical primitive for which texturing is enabled. To enable or disable one-dimensional texturing, call <seemfa marker="gl#enable/1"><c>gl:enable/1</c></seemfa> and <seemfa marker="gl#enable/1"><c>gl:disable/1</c></seemfa> with argument <c>?GL_TEXTURE_1D</c>. </p> + + <p><url href="https://www.khronos.org/registry/OpenGL-Refpages/gl4/html/glTexSubImage1D.xhtml">External documentation.</url></p></desc> + </func> + <func> <name name="texSubImage2D" arity="9" clause_i="1" since=""/> + <fsummary>specify a two-dimensional texture subimage</fsummary> + <desc> + <p> Texturing maps a portion of a specified texture image onto each graphical primitive for which texturing is enabled. </p> + + <p><url href="https://www.khronos.org/registry/OpenGL-Refpages/gl4/html/glTexSubImage2D.xhtml">External documentation.</url></p></desc> + </func> + <func> <name name="texSubImage3D" arity="11" clause_i="1" since=""/> - <fsummary/> - <desc><p>No documentation available.</p></desc> + <fsummary>specify a three-dimensional texture subimage</fsummary> + <desc> + <p> Texturing maps a portion of a specified texture image onto each graphical primitive for which texturing is enabled. </p> + + <p><url href="https://www.khronos.org/registry/OpenGL-Refpages/gl4/html/glTexSubImage3D.xhtml">External documentation.</url></p></desc> </func> <func> <name name="textureBarrier" arity="0" clause_i="1" since=""/> @@ -3942,16 +3503,6 @@ <p><url href="https://www.khronos.org/registry/OpenGL-Refpages/gl4/html/glTextureBarrier.xhtml">External documentation.</url></p></desc> </func> <func> - <name name="textureBuffer" arity="3" clause_i="1" since=""/> - <fsummary/> - <desc><p>No documentation available.</p></desc> - </func> - <func> - <name name="textureBufferRange" arity="5" clause_i="1" since=""/> - <fsummary/> - <desc><p>No documentation available.</p></desc> - </func> - <func> <name name="textureView" arity="8" clause_i="1" since=""/> <fsummary>initialize a texture as a data alias of another texture's data store</fsummary> <desc> @@ -3971,7 +3522,7 @@ <name name="transformFeedbackBufferRange" arity="5" clause_i="1" since=""/> <fsummary>bind a range within a buffer object to a transform feedback buffer object</fsummary> <desc> - <p><seemfa marker="gl#transformFeedbackBufferRange/5"><c>gl:transformFeedbackBufferRange/5</c></seemfa> binds a range of the buffer object <c>Buffer</c> represented by <c>Offset</c> and <c>Size</c> to the binding point at index <c>Index</c> of the transform feedback object <c>Xfb</c>. </p> + <p><seemfa marker="gl#transformFeedbackBufferRange/5"><c>gl:transformFeedbackBufferRange/5</c></seemfa> binds a range of the buffer object <c>Buffer</c> represented by <c>Offset</c> and <c>Size</c> to the binding point at index <c>Index</c> of the transform feedback object <c>Xfb</c>. </p> <p><url href="https://www.khronos.org/registry/OpenGL-Refpages/gl4/html/glTransformFeedbackBufferRange.xhtml">External documentation.</url></p></desc> </func> @@ -4050,86 +3601,6 @@ <p><url href="https://www.khronos.org/registry/OpenGL-Refpages/gl4/html/glUniform.xhtml">External documentation.</url></p></desc> </func> <func> - <name name="uniform1i64vARB" arity="2" clause_i="1" since=""/> - <fsummary/> - <desc><p>No documentation available.</p></desc> - </func> - <func> - <name name="uniform1i64ARB" arity="2" clause_i="1" since=""/> - <fsummary/> - <desc><p>No documentation available.</p></desc> - </func> - <func> - <name name="uniform1ui64vARB" arity="2" clause_i="1" since=""/> - <fsummary/> - <desc><p>No documentation available.</p></desc> - </func> - <func> - <name name="uniform1ui64ARB" arity="2" clause_i="1" since=""/> - <fsummary/> - <desc><p>No documentation available.</p></desc> - </func> - <func> - <name name="uniform2i64vARB" arity="2" clause_i="1" since=""/> - <fsummary/> - <desc><p>No documentation available.</p></desc> - </func> - <func> - <name name="uniform2i64ARB" arity="3" clause_i="1" since=""/> - <fsummary/> - <desc><p>No documentation available.</p></desc> - </func> - <func> - <name name="uniform2ui64vARB" arity="2" clause_i="1" since=""/> - <fsummary/> - <desc><p>No documentation available.</p></desc> - </func> - <func> - <name name="uniform2ui64ARB" arity="3" clause_i="1" since=""/> - <fsummary/> - <desc><p>No documentation available.</p></desc> - </func> - <func> - <name name="uniform3i64vARB" arity="2" clause_i="1" since=""/> - <fsummary/> - <desc><p>No documentation available.</p></desc> - </func> - <func> - <name name="uniform3i64ARB" arity="4" clause_i="1" since=""/> - <fsummary/> - <desc><p>No documentation available.</p></desc> - </func> - <func> - <name name="uniform3ui64vARB" arity="2" clause_i="1" since=""/> - <fsummary/> - <desc><p>No documentation available.</p></desc> - </func> - <func> - <name name="uniform3ui64ARB" arity="4" clause_i="1" since=""/> - <fsummary/> - <desc><p>No documentation available.</p></desc> - </func> - <func> - <name name="uniform4i64vARB" arity="2" clause_i="1" since=""/> - <fsummary/> - <desc><p>No documentation available.</p></desc> - </func> - <func> - <name name="uniform4i64ARB" arity="5" clause_i="1" since=""/> - <fsummary/> - <desc><p>No documentation available.</p></desc> - </func> - <func> - <name name="uniform4ui64vARB" arity="2" clause_i="1" since=""/> - <fsummary/> - <desc><p>No documentation available.</p></desc> - </func> - <func> - <name name="uniform4ui64ARB" arity="5" clause_i="1" since=""/> - <fsummary/> - <desc><p>No documentation available.</p></desc> - </func> - <func> <name name="uniformBlockBinding" arity="3" clause_i="1" since=""/> <fsummary>assign a binding point to an active uniform block</fsummary> <desc> @@ -4138,11 +3609,6 @@ <p><url href="https://www.khronos.org/registry/OpenGL-Refpages/gl4/html/glUniformBlockBinding.xhtml">External documentation.</url></p></desc> </func> <func> - <name name="uniformHandleui64ARB" arity="2" clause_i="1" since=""/> - <fsummary/> - <desc><p>No documentation available.</p></desc> - </func> - <func> <name name="uniformSubroutinesuiv" arity="2" clause_i="1" since=""/> <fsummary>load active subroutine uniforms</fsummary> <desc> @@ -4159,11 +3625,6 @@ <p><url href="https://www.khronos.org/registry/OpenGL-Refpages/gl4/html/glUseProgram.xhtml">External documentation.</url></p></desc> </func> <func> - <name name="useProgramObjectARB" arity="1" clause_i="1" since=""/> - <fsummary/> - <desc><p>No documentation available.</p></desc> - </func> - <func> <name name="useProgramStages" arity="3" clause_i="1" since=""/> <fsummary>bind stages of a program object to a program pipeline</fsummary> <desc> @@ -4180,11 +3641,6 @@ <p><url href="https://www.khronos.org/registry/OpenGL-Refpages/gl4/html/glValidateProgram.xhtml">External documentation.</url></p></desc> </func> <func> - <name name="validateProgramARB" arity="1" clause_i="1" since=""/> - <fsummary/> - <desc><p>No documentation available.</p></desc> - </func> - <func> <name name="validateProgramPipeline" arity="1" clause_i="1" since=""/> <fsummary>validate a program pipeline object against current GL state</fsummary> <desc> @@ -4224,31 +3680,6 @@ <p><url href="https://www.khronos.org/registry/OpenGL-Refpages/gl2.1/xhtml/glVertex.xml">External documentation.</url></p></desc> </func> <func> - <name name="vertexArrayAttribBinding" arity="3" clause_i="1" since=""/> - <fsummary/> - <desc><p>No documentation available.</p></desc> - </func> - <func> - <name name="vertexArrayAttribFormat" arity="6" clause_i="1" since=""/> - <fsummary/> - <desc><p>No documentation available.</p></desc> - </func> - <func> - <name name="vertexArrayAttribIFormat" arity="5" clause_i="1" since=""/> - <fsummary/> - <desc><p>No documentation available.</p></desc> - </func> - <func> - <name name="vertexArrayAttribLFormat" arity="5" clause_i="1" since=""/> - <fsummary/> - <desc><p>No documentation available.</p></desc> - </func> - <func> - <name name="vertexArrayBindingDivisor" arity="3" clause_i="1" since=""/> - <fsummary/> - <desc><p>No documentation available.</p></desc> - </func> - <func> <name name="vertexArrayElementBuffer" arity="2" clause_i="1" since=""/> <fsummary>configures element array buffer binding of a vertex array object</fsummary> <desc> @@ -4257,16 +3688,6 @@ <p><url href="https://www.khronos.org/registry/OpenGL-Refpages/gl4/html/glVertexArrayElementBuffer.xhtml">External documentation.</url></p></desc> </func> <func> - <name name="vertexArrayVertexBuffer" arity="5" clause_i="1" since=""/> - <fsummary/> - <desc><p>No documentation available.</p></desc> - </func> - <func> - <name name="vertexArrayVertexBuffers" arity="5" clause_i="1" since=""/> - <fsummary/> - <desc><p>No documentation available.</p></desc> - </func> - <func> <name name="vertexAttrib1d" arity="2" clause_i="1" since=""/> <name name="vertexAttrib1dv" arity="2" clause_i="1" since=""/> <name name="vertexAttrib1f" arity="2" clause_i="1" since=""/> @@ -4323,6 +3744,14 @@ <name name="vertexAttribI4ui" arity="5" clause_i="1" since=""/> <name name="vertexAttribI4uiv" arity="2" clause_i="1" since=""/> <name name="vertexAttribI4usv" arity="2" clause_i="1" since=""/> + <name name="vertexAttribL1d" arity="2" clause_i="1" since=""/> + <name name="vertexAttribL1dv" arity="2" clause_i="1" since=""/> + <name name="vertexAttribL2d" arity="3" clause_i="1" since=""/> + <name name="vertexAttribL2dv" arity="2" clause_i="1" since=""/> + <name name="vertexAttribL3d" arity="4" clause_i="1" since=""/> + <name name="vertexAttribL3dv" arity="2" clause_i="1" since=""/> + <name name="vertexAttribL4d" arity="5" clause_i="1" since=""/> + <name name="vertexAttribL4dv" arity="2" clause_i="1" since=""/> <fsummary>Specifies the value of a generic vertex attribute</fsummary> <desc> <p>The <seemfa marker="gl#vertexAttrib1d/2"><c>gl:vertexAttrib()</c></seemfa> family of entry points allows an application to pass generic vertex attributes in numbered locations.</p> @@ -4330,10 +3759,11 @@ <p><url href="https://www.khronos.org/registry/OpenGL-Refpages/gl4/html/glVertexAttrib.xhtml">External documentation.</url></p></desc> </func> <func> + <name name="vertexArrayAttribBinding" arity="3" clause_i="1" since=""/> <name name="vertexAttribBinding" arity="2" clause_i="1" since=""/> <fsummary>associate a vertex attribute and a vertex buffer binding for a vertex array object</fsummary> <desc> - <p><seemfa marker="gl#vertexAttribBinding/2"><c>gl:vertexAttribBinding/2</c></seemfa> and <seemfa marker="gl#vertexArrayAttribBinding/3"><c>gl:vertexArrayAttribBinding/3</c></seemfa> establishes an association between the generic vertex attribute of a vertex array object whose index is given by <c>Attribindex</c>, and a vertex buffer binding whose index is given by <c>Bindingindex</c>. For <seemfa marker="gl#vertexAttribBinding/2"><c>gl:vertexAttribBinding/2</c></seemfa>, the vertex array object affected is that currently bound. For <seemfa marker="gl#vertexArrayAttribBinding/3"><c>gl:vertexArrayAttribBinding/3</c></seemfa>, <c>Vaobj</c> is the name of the vertex array object. </p> + <p><seemfa marker="gl#vertexAttribBinding/2"><c>gl:vertexAttribBinding/2</c></seemfa> and <seemfa marker="gl#vertexAttribBinding/2"><c>gl:vertexArrayAttribBinding/3</c></seemfa> establishes an association between the generic vertex attribute of a vertex array object whose index is given by <c>Attribindex</c>, and a vertex buffer binding whose index is given by <c>Bindingindex</c>. For <seemfa marker="gl#vertexAttribBinding/2"><c>gl:vertexAttribBinding/2</c></seemfa>, the vertex array object affected is that currently bound. For <seemfa marker="gl#vertexAttribBinding/2"><c>gl:vertexArrayAttribBinding/3</c></seemfa>, <c>Vaobj</c> is the name of the vertex array object. </p> <p><url href="https://www.khronos.org/registry/OpenGL-Refpages/gl4/html/glVertexAttribBinding.xhtml">External documentation.</url></p></desc> </func> @@ -4346,67 +3776,38 @@ <p><url href="https://www.khronos.org/registry/OpenGL-Refpages/gl4/html/glVertexAttribDivisor.xhtml">External documentation.</url></p></desc> </func> <func> + <name name="vertexArrayAttribFormat" arity="6" clause_i="1" since=""/> + <name name="vertexArrayAttribIFormat" arity="5" clause_i="1" since=""/> + <name name="vertexArrayAttribLFormat" arity="5" clause_i="1" since=""/> <name name="vertexAttribFormat" arity="5" clause_i="1" since=""/> - <fsummary>specify the organization of vertex arrays</fsummary> - <desc> - <p><seemfa marker="gl#vertexAttribFormat/5"><c>gl:vertexAttribFormat/5</c></seemfa>, <seemfa marker="gl#vertexAttribIFormat/4"><c>gl:vertexAttribIFormat/4</c></seemfa> and <seemfa marker="gl#vertexAttribLFormat/4"><c>gl:vertexAttribLFormat/4</c></seemfa>, as well as <seemfa marker="gl#vertexArrayAttribFormat/6"><c>gl:vertexArrayAttribFormat/6</c></seemfa>, <seemfa marker="gl#vertexArrayAttribIFormat/5"><c>gl:vertexArrayAttribIFormat/5</c></seemfa> and <seemfa marker="gl#vertexArrayAttribLFormat/5"><c>gl:vertexArrayAttribLFormat/5</c></seemfa> specify the organization of data in vertex arrays. The first three calls operate on the bound vertex array object, whereas the last three ones modify the state of a vertex array object with ID <c>Vaobj</c>. <c>Attribindex</c> specifies the index of the generic vertex attribute array whose data layout is being described, and must be less than the value of <c>?GL_MAX_VERTEX_ATTRIBS</c>.</p> - - <p><url href="https://www.khronos.org/registry/OpenGL-Refpages/gl4/html/glVertexAttribFormat.xhtml">External documentation.</url></p></desc> - </func> - <func> <name name="vertexAttribIFormat" arity="4" clause_i="1" since=""/> - <fsummary/> - <desc><p>No documentation available.</p></desc> - </func> - <func> <name name="vertexAttribIPointer" arity="5" clause_i="1" since=""/> - <fsummary/> - <desc><p>No documentation available.</p></desc> - </func> - <func> - <name name="vertexAttribL1d" arity="2" clause_i="1" since=""/> - <name name="vertexAttribL1dv" arity="2" clause_i="1" since=""/> - <name name="vertexAttribL2d" arity="3" clause_i="1" since=""/> - <name name="vertexAttribL2dv" arity="2" clause_i="1" since=""/> - <name name="vertexAttribL3d" arity="4" clause_i="1" since=""/> - <name name="vertexAttribL3dv" arity="2" clause_i="1" since=""/> - <name name="vertexAttribL4d" arity="5" clause_i="1" since=""/> - <name name="vertexAttribL4dv" arity="2" clause_i="1" since=""/> - <fsummary/> - <desc><p>No documentation available.</p></desc> - </func> - <func> <name name="vertexAttribLFormat" arity="4" clause_i="1" since=""/> - <fsummary/> - <desc><p>No documentation available.</p></desc> - </func> - <func> <name name="vertexAttribLPointer" arity="5" clause_i="1" since=""/> - <fsummary/> - <desc><p>No documentation available.</p></desc> + <fsummary>specify the organization of vertex arrays</fsummary> + <desc> + <p><seemfa marker="gl#vertexAttribFormat/5"><c>gl:vertexAttribFormat/5</c></seemfa>, <seemfa marker="gl#vertexAttribIPointer/5"><c>gl:vertexAttribIFormat/4</c></seemfa> and <seemfa marker="gl#vertexAttribIPointer/5"><c>gl:vertexAttribLFormat/4</c></seemfa>, as well as <seemfa marker="gl#vertexAttribIPointer/5"><c>gl:vertexArrayAttribFormat/6</c></seemfa>, <seemfa marker="gl#vertexAttribIPointer/5"><c>gl:vertexArrayAttribIFormat/5</c></seemfa> and <seemfa marker="gl#vertexAttribIPointer/5"><c>gl:vertexArrayAttribLFormat/5</c></seemfa> specify the organization of data in vertex arrays. The first three calls operate on the bound vertex array object, whereas the last three ones modify the state of a vertex array object with ID <c>Vaobj</c>. <c>Attribindex</c> specifies the index of the generic vertex attribute array whose data layout is being described, and must be less than the value of <c>?GL_MAX_VERTEX_ATTRIBS</c>.</p> + + <p><url href="https://www.khronos.org/registry/OpenGL-Refpages/gl4/html/glVertexAttribFormat.xhtml">External documentation.</url></p></desc> </func> <func> <name name="vertexAttribPointer" arity="6" clause_i="1" since=""/> <fsummary>define an array of generic vertex attribute data</fsummary> <desc> - <p><seemfa marker="gl#vertexAttribPointer/6"><c>gl:vertexAttribPointer/6</c></seemfa>, <seemfa marker="gl#vertexAttribIPointer/5"><c>gl:vertexAttribIPointer/5</c></seemfa> and <seemfa marker="gl#vertexAttribLPointer/5"><c>gl:vertexAttribLPointer/5</c></seemfa> specify the location and data format of the array of generic vertex attributes at index <c>Index</c> to use when rendering. <c>Size</c> specifies the number of components per attribute and must be 1, 2, 3, 4, or <c>?GL_BGRA</c>. <c>Type</c> specifies the data type of each component, and <c>Stride</c> specifies the byte stride from one attribute to the next, allowing vertices and attributes to be packed into a single array or stored in separate arrays. </p> + <p><seemfa marker="gl#vertexAttribPointer/6"><c>gl:vertexAttribPointer/6</c></seemfa>, <seemfa marker="gl#vertexAttribIPointer/5"><c>gl:vertexAttribIPointer/5</c></seemfa> and <seemfa marker="gl#vertexAttribIPointer/5"><c>gl:vertexAttribLPointer/5</c></seemfa> specify the location and data format of the array of generic vertex attributes at index <c>Index</c> to use when rendering. <c>Size</c> specifies the number of components per attribute and must be 1, 2, 3, 4, or <c>?GL_BGRA</c>. <c>Type</c> specifies the data type of each component, and <c>Stride</c> specifies the byte stride from one attribute to the next, allowing vertices and attributes to be packed into a single array or stored in separate arrays. </p> <p><url href="https://www.khronos.org/registry/OpenGL-Refpages/gl4/html/glVertexAttribPointer.xhtml">External documentation.</url></p></desc> </func> <func> + <name name="vertexArrayBindingDivisor" arity="3" clause_i="1" since=""/> <name name="vertexBindingDivisor" arity="2" clause_i="1" since=""/> <fsummary>modify the rate at which generic vertex attributes advance</fsummary> <desc> - <p><seemfa marker="gl#vertexBindingDivisor/2"><c>gl:vertexBindingDivisor/2</c></seemfa> and <seemfa marker="gl#vertexArrayBindingDivisor/3"><c>gl:vertexArrayBindingDivisor/3</c></seemfa> modify the rate at which generic vertex attributes advance when rendering multiple instances of primitives in a single draw command. If <c>Divisor</c> is zero, the attributes using the buffer bound to <c>Bindingindex</c> advance once per vertex. If <c>Divisor</c> is non-zero, the attributes advance once per <c>Divisor</c> instances of the set(s) of vertices being rendered. An attribute is referred to as <c>instanced</c> if the corresponding <c>Divisor</c> value is non-zero.</p> + <p><seemfa marker="gl#vertexBindingDivisor/2"><c>gl:vertexBindingDivisor/2</c></seemfa> and <seemfa marker="gl#vertexBindingDivisor/2"><c>gl:vertexArrayBindingDivisor/3</c></seemfa> modify the rate at which generic vertex attributes advance when rendering multiple instances of primitives in a single draw command. If <c>Divisor</c> is zero, the attributes using the buffer bound to <c>Bindingindex</c> advance once per vertex. If <c>Divisor</c> is non-zero, the attributes advance once per <c>Divisor</c> instances of the set(s) of vertices being rendered. An attribute is referred to as <c>instanced</c> if the corresponding <c>Divisor</c> value is non-zero.</p> <p><url href="https://www.khronos.org/registry/OpenGL-Refpages/gl4/html/glVertexBindingDivisor.xhtml">External documentation.</url></p></desc> </func> <func> - <name name="vertexBlendARB" arity="1" clause_i="1" since=""/> - <fsummary/> - <desc><p>No documentation available.</p></desc> - </func> - <func> <name name="vertexPointer" arity="4" clause_i="1" since=""/> <fsummary>define an array of vertex data</fsummary> <desc> @@ -4424,8 +3825,11 @@ </func> <func> <name name="viewportArrayv" arity="2" clause_i="1" since=""/> - <fsummary/> - <desc><p>No documentation available.</p></desc> + <fsummary>set multiple viewports</fsummary> + <desc> + <p><seemfa marker="gl#viewportArrayv/2"><c>gl:viewportArrayv/2</c></seemfa> specifies the parameters for multiple viewports simulataneously. <c>First</c> specifies the index of the first viewport to modify and <c>Count</c> specifies the number of viewports to modify. <c>First</c> must be less than the value of <c>?GL_MAX_VIEWPORTS</c>, and <c>First</c> + <c>Count</c> must be less than or equal to the value of <c>?GL_MAX_VIEWPORTS</c>. Viewports whose indices lie outside the range [<c>First</c>, <c>First</c> + <c>Count</c>) are not modified. <c>V</c> contains the address of an array of floating point values specifying the left ( x), bottom ( y), width ( w), and height ( h) of each viewport, in that order. x and y give the location of the viewport's lower left corner, and w and h give the width and height of the viewport, respectively. The viewport specifies the affine transformation of x and y from normalized device coordinates to window coordinates. Let (x nd y nd) be normalized device coordinates. Then the window coordinates (x w y w) are computed as follows: </p> + + <p><url href="https://www.khronos.org/registry/OpenGL-Refpages/gl4/html/glViewportArray.xhtml">External documentation.</url></p></desc> </func> <func> <name name="viewportIndexedf" arity="5" clause_i="1" since=""/> @@ -4446,18 +3850,6 @@ <p><url href="https://www.khronos.org/registry/OpenGL-Refpages/gl4/html/glWaitSync.xhtml">External documentation.</url></p></desc> </func> <func> - <name name="weightbvARB" arity="1" clause_i="1" since=""/> - <name name="weightdvARB" arity="1" clause_i="1" since=""/> - <name name="weightfvARB" arity="1" clause_i="1" since=""/> - <name name="weightivARB" arity="1" clause_i="1" since=""/> - <name name="weightsvARB" arity="1" clause_i="1" since=""/> - <name name="weightubvARB" arity="1" clause_i="1" since=""/> - <name name="weightuivARB" arity="1" clause_i="1" since=""/> - <name name="weightusvARB" arity="1" clause_i="1" since=""/> - <fsummary/> - <desc><p>No documentation available.</p></desc> - </func> - <func> <name name="windowPos2d" arity="2" clause_i="1" since=""/> <name name="windowPos2dv" arity="1" clause_i="1" since=""/> <name name="windowPos2f" arity="2" clause_i="1" since=""/> diff --git a/make/otp_version_tickets_in_merge b/make/otp_version_tickets_in_merge index 553ca835d2..03a6d79472 100644 --- a/make/otp_version_tickets_in_merge +++ b/make/otp_version_tickets_in_merge @@ -1,8 +1 @@ -OTP-17393 -OTP-17657 -OTP-17658 -OTP-17659 -OTP-17666 -OTP-17668 -OTP-17670 -OTP-17672 +OTP-17764 diff --git a/otp_versions.table b/otp_versions.table index 54841f5bf3..1a5dbfab8a 100644 --- a/otp_versions.table +++ b/otp_versions.table @@ -1,3 +1,8 @@ +OTP-24.1.7 : ssh-4.12.5 # asn1-5.0.17 common_test-1.21 compiler-8.0.3 crypto-5.0.4 debugger-5.2 dialyzer-4.4.2 diameter-2.2.4 edoc-1.0.1 eldap-1.2.9 erl_docgen-1.1.2 erl_interface-5.1 erts-12.1.5 et-1.6.5 eunit-2.7 ftp-1.1 inets-7.4.2 jinterface-1.12.1 kernel-8.1.3 megaco-4.1 mnesia-4.20 observer-2.10 odbc-2.13.5 os_mon-2.7.1 parsetools-2.3.1 public_key-1.11.3 reltool-0.9 runtime_tools-1.17 sasl-4.1 snmp-5.10.1 ssl-10.5.3 stdlib-3.16.1 syntax_tools-2.6 tftp-1.0.3 tools-3.5.1 wx-2.1 xmerl-1.3.28 : +OTP-24.1.6 : ssl-10.5.3 # asn1-5.0.17 common_test-1.21 compiler-8.0.3 crypto-5.0.4 debugger-5.2 dialyzer-4.4.2 diameter-2.2.4 edoc-1.0.1 eldap-1.2.9 erl_docgen-1.1.2 erl_interface-5.1 erts-12.1.5 et-1.6.5 eunit-2.7 ftp-1.1 inets-7.4.2 jinterface-1.12.1 kernel-8.1.3 megaco-4.1 mnesia-4.20 observer-2.10 odbc-2.13.5 os_mon-2.7.1 parsetools-2.3.1 public_key-1.11.3 reltool-0.9 runtime_tools-1.17 sasl-4.1 snmp-5.10.1 ssh-4.12.4 stdlib-3.16.1 syntax_tools-2.6 tftp-1.0.3 tools-3.5.1 wx-2.1 xmerl-1.3.28 : +OTP-24.1.5 : erts-12.1.5 kernel-8.1.3 # asn1-5.0.17 common_test-1.21 compiler-8.0.3 crypto-5.0.4 debugger-5.2 dialyzer-4.4.2 diameter-2.2.4 edoc-1.0.1 eldap-1.2.9 erl_docgen-1.1.2 erl_interface-5.1 et-1.6.5 eunit-2.7 ftp-1.1 inets-7.4.2 jinterface-1.12.1 megaco-4.1 mnesia-4.20 observer-2.10 odbc-2.13.5 os_mon-2.7.1 parsetools-2.3.1 public_key-1.11.3 reltool-0.9 runtime_tools-1.17 sasl-4.1 snmp-5.10.1 ssh-4.12.4 ssl-10.5.2 stdlib-3.16.1 syntax_tools-2.6 tftp-1.0.3 tools-3.5.1 wx-2.1 xmerl-1.3.28 : +OTP-24.1.4 : erts-12.1.4 # asn1-5.0.17 common_test-1.21 compiler-8.0.3 crypto-5.0.4 debugger-5.2 dialyzer-4.4.2 diameter-2.2.4 edoc-1.0.1 eldap-1.2.9 erl_docgen-1.1.2 erl_interface-5.1 et-1.6.5 eunit-2.7 ftp-1.1 inets-7.4.2 jinterface-1.12.1 kernel-8.1.2 megaco-4.1 mnesia-4.20 observer-2.10 odbc-2.13.5 os_mon-2.7.1 parsetools-2.3.1 public_key-1.11.3 reltool-0.9 runtime_tools-1.17 sasl-4.1 snmp-5.10.1 ssh-4.12.4 ssl-10.5.2 stdlib-3.16.1 syntax_tools-2.6 tftp-1.0.3 tools-3.5.1 wx-2.1 xmerl-1.3.28 : +OTP-24.1.3 : erts-12.1.3 ssl-10.5.2 # asn1-5.0.17 common_test-1.21 compiler-8.0.3 crypto-5.0.4 debugger-5.2 dialyzer-4.4.2 diameter-2.2.4 edoc-1.0.1 eldap-1.2.9 erl_docgen-1.1.2 erl_interface-5.1 et-1.6.5 eunit-2.7 ftp-1.1 inets-7.4.2 jinterface-1.12.1 kernel-8.1.2 megaco-4.1 mnesia-4.20 observer-2.10 odbc-2.13.5 os_mon-2.7.1 parsetools-2.3.1 public_key-1.11.3 reltool-0.9 runtime_tools-1.17 sasl-4.1 snmp-5.10.1 ssh-4.12.4 stdlib-3.16.1 syntax_tools-2.6 tftp-1.0.3 tools-3.5.1 wx-2.1 xmerl-1.3.28 : OTP-24.1.2 : crypto-5.0.4 erts-12.1.2 kernel-8.1.2 public_key-1.11.3 ssl-10.5.1 # asn1-5.0.17 common_test-1.21 compiler-8.0.3 debugger-5.2 dialyzer-4.4.2 diameter-2.2.4 edoc-1.0.1 eldap-1.2.9 erl_docgen-1.1.2 erl_interface-5.1 et-1.6.5 eunit-2.7 ftp-1.1 inets-7.4.2 jinterface-1.12.1 megaco-4.1 mnesia-4.20 observer-2.10 odbc-2.13.5 os_mon-2.7.1 parsetools-2.3.1 reltool-0.9 runtime_tools-1.17 sasl-4.1 snmp-5.10.1 ssh-4.12.4 stdlib-3.16.1 syntax_tools-2.6 tftp-1.0.3 tools-3.5.1 wx-2.1 xmerl-1.3.28 : OTP-24.1.1 : erts-12.1.1 kernel-8.1.1 snmp-5.10.1 stdlib-3.16.1 # asn1-5.0.17 common_test-1.21 compiler-8.0.3 crypto-5.0.3 debugger-5.2 dialyzer-4.4.2 diameter-2.2.4 edoc-1.0.1 eldap-1.2.9 erl_docgen-1.1.2 erl_interface-5.1 et-1.6.5 eunit-2.7 ftp-1.1 inets-7.4.2 jinterface-1.12.1 megaco-4.1 mnesia-4.20 observer-2.10 odbc-2.13.5 os_mon-2.7.1 parsetools-2.3.1 public_key-1.11.2 reltool-0.9 runtime_tools-1.17 sasl-4.1 ssh-4.12.4 ssl-10.5 syntax_tools-2.6 tftp-1.0.3 tools-3.5.1 wx-2.1 xmerl-1.3.28 : OTP-24.1 : asn1-5.0.17 common_test-1.21 compiler-8.0.3 crypto-5.0.3 debugger-5.2 dialyzer-4.4.2 edoc-1.0.1 erl_docgen-1.1.2 erl_interface-5.1 erts-12.1 eunit-2.7 inets-7.4.2 jinterface-1.12.1 kernel-8.1 megaco-4.1 mnesia-4.20 observer-2.10 os_mon-2.7.1 parsetools-2.3.1 public_key-1.11.2 runtime_tools-1.17 snmp-5.10 ssh-4.12.4 ssl-10.5 stdlib-3.16 tools-3.5.1 wx-2.1 # diameter-2.2.4 eldap-1.2.9 et-1.6.5 ftp-1.1 odbc-2.13.5 reltool-0.9 sasl-4.1 syntax_tools-2.6 tftp-1.0.3 xmerl-1.3.28 : @@ -8,6 +13,8 @@ OTP-24.0.3 : compiler-8.0.2 dialyzer-4.4.1 erts-12.0.3 inets-7.4.1 ssh-4.12.3 # OTP-24.0.2 : compiler-8.0.1 crypto-5.0.2 erl_docgen-1.1.1 erts-12.0.2 kernel-8.0.1 ssh-4.12.2 ssl-10.4.1 stdlib-3.15.1 # asn1-5.0.16 common_test-1.20.4 debugger-5.1 dialyzer-4.4 diameter-2.2.4 edoc-1.0 eldap-1.2.9 erl_interface-5.0.1 et-1.6.5 eunit-2.6.1 ftp-1.1 inets-7.4 jinterface-1.12 megaco-4.0.1 mnesia-4.19.1 observer-2.9.6 odbc-2.13.5 os_mon-2.7 parsetools-2.3 public_key-1.11 reltool-0.9 runtime_tools-1.16.2 sasl-4.1 snmp-5.9.1 syntax_tools-2.6 tftp-1.0.3 tools-3.5 wx-2.0.1 xmerl-1.3.28 : OTP-24.0.1 : common_test-1.20.4 crypto-5.0.1 erl_interface-5.0.1 erts-12.0.1 megaco-4.0.1 odbc-2.13.5 snmp-5.9.1 ssh-4.12.1 wx-2.0.1 # asn1-5.0.16 compiler-8.0 debugger-5.1 dialyzer-4.4 diameter-2.2.4 edoc-1.0 eldap-1.2.9 erl_docgen-1.1 et-1.6.5 eunit-2.6.1 ftp-1.1 inets-7.4 jinterface-1.12 kernel-8.0 mnesia-4.19.1 observer-2.9.6 os_mon-2.7 parsetools-2.3 public_key-1.11 reltool-0.9 runtime_tools-1.16.2 sasl-4.1 ssl-10.4 stdlib-3.15 syntax_tools-2.6 tftp-1.0.3 tools-3.5 xmerl-1.3.28 : OTP-24.0 : asn1-5.0.16 common_test-1.20.3 compiler-8.0 crypto-5.0 debugger-5.1 dialyzer-4.4 edoc-1.0 erl_docgen-1.1 erl_interface-5.0 erts-12.0 et-1.6.5 eunit-2.6.1 ftp-1.1 inets-7.4 jinterface-1.12 kernel-8.0 megaco-4.0 mnesia-4.19.1 observer-2.9.6 odbc-2.13.4 os_mon-2.7 parsetools-2.3 public_key-1.11 reltool-0.9 runtime_tools-1.16.2 sasl-4.1 snmp-5.9 ssh-4.12 ssl-10.4 stdlib-3.15 syntax_tools-2.6 tftp-1.0.3 tools-3.5 wx-2.0 xmerl-1.3.28 # diameter-2.2.4 eldap-1.2.9 : +OTP-23.3.4.9 : erts-11.2.2.8 # asn1-5.0.15.1 common_test-1.20.2.2 compiler-7.6.9.1 crypto-4.9.0.2 debugger-5.0 dialyzer-4.3.1.1 diameter-2.2.4 edoc-0.12 eldap-1.2.9 erl_docgen-1.0.2 erl_interface-4.0.3.1 et-1.6.4 eunit-2.6 ftp-1.0.5 hipe-4.0.1 inets-7.3.2.2 jinterface-1.11.1 kernel-7.3.1.4 megaco-3.19.5.1 mnesia-4.19 observer-2.9.5 odbc-2.13.3.1 os_mon-2.6.1 parsetools-2.2 public_key-1.10.0.1 reltool-0.8 runtime_tools-1.16.1 sasl-4.0.2 snmp-5.8.0.1 ssh-4.11.1.4 ssl-10.3.1.2 stdlib-3.14.2.2 syntax_tools-2.5 tftp-1.0.2 tools-3.4.4 wx-1.9.3.1 xmerl-1.3.27 : +OTP-23.3.4.8 : erts-11.2.2.7 ssh-4.11.1.4 # asn1-5.0.15.1 common_test-1.20.2.2 compiler-7.6.9.1 crypto-4.9.0.2 debugger-5.0 dialyzer-4.3.1.1 diameter-2.2.4 edoc-0.12 eldap-1.2.9 erl_docgen-1.0.2 erl_interface-4.0.3.1 et-1.6.4 eunit-2.6 ftp-1.0.5 hipe-4.0.1 inets-7.3.2.2 jinterface-1.11.1 kernel-7.3.1.4 megaco-3.19.5.1 mnesia-4.19 observer-2.9.5 odbc-2.13.3.1 os_mon-2.6.1 parsetools-2.2 public_key-1.10.0.1 reltool-0.8 runtime_tools-1.16.1 sasl-4.0.2 snmp-5.8.0.1 ssl-10.3.1.2 stdlib-3.14.2.2 syntax_tools-2.5 tftp-1.0.2 tools-3.4.4 wx-1.9.3.1 xmerl-1.3.27 : OTP-23.3.4.7 : erts-11.2.2.6 inets-7.3.2.2 kernel-7.3.1.4 # asn1-5.0.15.1 common_test-1.20.2.2 compiler-7.6.9.1 crypto-4.9.0.2 debugger-5.0 dialyzer-4.3.1.1 diameter-2.2.4 edoc-0.12 eldap-1.2.9 erl_docgen-1.0.2 erl_interface-4.0.3.1 et-1.6.4 eunit-2.6 ftp-1.0.5 hipe-4.0.1 jinterface-1.11.1 megaco-3.19.5.1 mnesia-4.19 observer-2.9.5 odbc-2.13.3.1 os_mon-2.6.1 parsetools-2.2 public_key-1.10.0.1 reltool-0.8 runtime_tools-1.16.1 sasl-4.0.2 snmp-5.8.0.1 ssh-4.11.1.3 ssl-10.3.1.2 stdlib-3.14.2.2 syntax_tools-2.5 tftp-1.0.2 tools-3.4.4 wx-1.9.3.1 xmerl-1.3.27 : OTP-23.3.4.6 : erts-11.2.2.5 kernel-7.3.1.3 # asn1-5.0.15.1 common_test-1.20.2.2 compiler-7.6.9.1 crypto-4.9.0.2 debugger-5.0 dialyzer-4.3.1.1 diameter-2.2.4 edoc-0.12 eldap-1.2.9 erl_docgen-1.0.2 erl_interface-4.0.3.1 et-1.6.4 eunit-2.6 ftp-1.0.5 hipe-4.0.1 inets-7.3.2.1 jinterface-1.11.1 megaco-3.19.5.1 mnesia-4.19 observer-2.9.5 odbc-2.13.3.1 os_mon-2.6.1 parsetools-2.2 public_key-1.10.0.1 reltool-0.8 runtime_tools-1.16.1 sasl-4.0.2 snmp-5.8.0.1 ssh-4.11.1.3 ssl-10.3.1.2 stdlib-3.14.2.2 syntax_tools-2.5 tftp-1.0.2 tools-3.4.4 wx-1.9.3.1 xmerl-1.3.27 : OTP-23.3.4.5 : asn1-5.0.15.1 common_test-1.20.2.2 erts-11.2.2.4 public_key-1.10.0.1 ssl-10.3.1.2 stdlib-3.14.2.2 # compiler-7.6.9.1 crypto-4.9.0.2 debugger-5.0 dialyzer-4.3.1.1 diameter-2.2.4 edoc-0.12 eldap-1.2.9 erl_docgen-1.0.2 erl_interface-4.0.3.1 et-1.6.4 eunit-2.6 ftp-1.0.5 hipe-4.0.1 inets-7.3.2.1 jinterface-1.11.1 kernel-7.3.1.2 megaco-3.19.5.1 mnesia-4.19 observer-2.9.5 odbc-2.13.3.1 os_mon-2.6.1 parsetools-2.2 reltool-0.8 runtime_tools-1.16.1 sasl-4.0.2 snmp-5.8.0.1 ssh-4.11.1.3 syntax_tools-2.5 tftp-1.0.2 tools-3.4.4 wx-1.9.3.1 xmerl-1.3.27 : @@ -44,6 +51,8 @@ OTP-23.0.3 : compiler-7.6.2 erts-11.0.3 # asn1-5.0.13 common_test-1.19 crypto-4. OTP-23.0.2 : erts-11.0.2 megaco-3.19.1 # asn1-5.0.13 common_test-1.19 compiler-7.6.1 crypto-4.7 debugger-5.0 dialyzer-4.2 diameter-2.2.3 edoc-0.12 eldap-1.2.8 erl_docgen-1.0 erl_interface-4.0 et-1.6.4 eunit-2.5 ftp-1.0.4 hipe-4.0 inets-7.2 jinterface-1.11 kernel-7.0 mnesia-4.17 observer-2.9.4 odbc-2.13 os_mon-2.5.2 parsetools-2.2 public_key-1.8 reltool-0.8 runtime_tools-1.15 sasl-4.0 snmp-5.6 ssh-4.10 ssl-10.0 stdlib-3.13 syntax_tools-2.3 tftp-1.0.2 tools-3.4 wx-1.9.1 xmerl-1.3.25 : OTP-23.0.1 : compiler-7.6.1 erts-11.0.1 # asn1-5.0.13 common_test-1.19 crypto-4.7 debugger-5.0 dialyzer-4.2 diameter-2.2.3 edoc-0.12 eldap-1.2.8 erl_docgen-1.0 erl_interface-4.0 et-1.6.4 eunit-2.5 ftp-1.0.4 hipe-4.0 inets-7.2 jinterface-1.11 kernel-7.0 megaco-3.19 mnesia-4.17 observer-2.9.4 odbc-2.13 os_mon-2.5.2 parsetools-2.2 public_key-1.8 reltool-0.8 runtime_tools-1.15 sasl-4.0 snmp-5.6 ssh-4.10 ssl-10.0 stdlib-3.13 syntax_tools-2.3 tftp-1.0.2 tools-3.4 wx-1.9.1 xmerl-1.3.25 : OTP-23.0 : asn1-5.0.13 common_test-1.19 compiler-7.6 crypto-4.7 debugger-5.0 dialyzer-4.2 edoc-0.12 erl_docgen-1.0 erl_interface-4.0 erts-11.0 eunit-2.5 hipe-4.0 inets-7.2 jinterface-1.11 kernel-7.0 megaco-3.19 mnesia-4.17 observer-2.9.4 odbc-2.13 os_mon-2.5.2 parsetools-2.2 public_key-1.8 runtime_tools-1.15 sasl-4.0 snmp-5.6 ssh-4.10 ssl-10.0 stdlib-3.13 syntax_tools-2.3 tools-3.4 wx-1.9.1 xmerl-1.3.25 # diameter-2.2.3 eldap-1.2.8 et-1.6.4 ftp-1.0.4 reltool-0.8 tftp-1.0.2 : +OTP-22.3.4.23 : erts-10.7.2.15 # asn1-5.0.12 common_test-1.18.2.1 compiler-7.5.4.3 crypto-4.6.5.4 debugger-4.2.8 dialyzer-4.1.1 diameter-2.2.3 edoc-0.11 eldap-1.2.8 erl_docgen-0.11 erl_interface-3.13.2.2 et-1.6.4 eunit-2.4.1 ftp-1.0.4.1 hipe-3.19.3 inets-7.1.3.3 jinterface-1.10.1 kernel-6.5.2.4 megaco-3.18.8.4 mnesia-4.16.3.1 observer-2.9.3 odbc-2.12.4.1 os_mon-2.5.1.1 parsetools-2.1.8 public_key-1.7.2 reltool-0.8 runtime_tools-1.14.0.1 sasl-3.4.2 snmp-5.5.0.5 ssh-4.9.1.4 ssl-9.6.2.3 stdlib-3.12.1.2 syntax_tools-2.2.1 tftp-1.0.2 tools-3.3.1.1 wx-1.9.0.1 xmerl-1.3.24 : +OTP-22.3.4.22 : erts-10.7.2.14 ssh-4.9.1.4 # asn1-5.0.12 common_test-1.18.2.1 compiler-7.5.4.3 crypto-4.6.5.4 debugger-4.2.8 dialyzer-4.1.1 diameter-2.2.3 edoc-0.11 eldap-1.2.8 erl_docgen-0.11 erl_interface-3.13.2.2 et-1.6.4 eunit-2.4.1 ftp-1.0.4.1 hipe-3.19.3 inets-7.1.3.3 jinterface-1.10.1 kernel-6.5.2.4 megaco-3.18.8.4 mnesia-4.16.3.1 observer-2.9.3 odbc-2.12.4.1 os_mon-2.5.1.1 parsetools-2.1.8 public_key-1.7.2 reltool-0.8 runtime_tools-1.14.0.1 sasl-3.4.2 snmp-5.5.0.5 ssl-9.6.2.3 stdlib-3.12.1.2 syntax_tools-2.2.1 tftp-1.0.2 tools-3.3.1.1 wx-1.9.0.1 xmerl-1.3.24 : OTP-22.3.4.21 : erts-10.7.2.13 kernel-6.5.2.4 stdlib-3.12.1.2 # asn1-5.0.12 common_test-1.18.2.1 compiler-7.5.4.3 crypto-4.6.5.4 debugger-4.2.8 dialyzer-4.1.1 diameter-2.2.3 edoc-0.11 eldap-1.2.8 erl_docgen-0.11 erl_interface-3.13.2.2 et-1.6.4 eunit-2.4.1 ftp-1.0.4.1 hipe-3.19.3 inets-7.1.3.3 jinterface-1.10.1 megaco-3.18.8.4 mnesia-4.16.3.1 observer-2.9.3 odbc-2.12.4.1 os_mon-2.5.1.1 parsetools-2.1.8 public_key-1.7.2 reltool-0.8 runtime_tools-1.14.0.1 sasl-3.4.2 snmp-5.5.0.5 ssh-4.9.1.3 ssl-9.6.2.3 syntax_tools-2.2.1 tftp-1.0.2 tools-3.3.1.1 wx-1.9.0.1 xmerl-1.3.24 : OTP-22.3.4.20 : crypto-4.6.5.4 erts-10.7.2.12 kernel-6.5.2.3 stdlib-3.12.1.1 # asn1-5.0.12 common_test-1.18.2.1 compiler-7.5.4.3 debugger-4.2.8 dialyzer-4.1.1 diameter-2.2.3 edoc-0.11 eldap-1.2.8 erl_docgen-0.11 erl_interface-3.13.2.2 et-1.6.4 eunit-2.4.1 ftp-1.0.4.1 hipe-3.19.3 inets-7.1.3.3 jinterface-1.10.1 megaco-3.18.8.4 mnesia-4.16.3.1 observer-2.9.3 odbc-2.12.4.1 os_mon-2.5.1.1 parsetools-2.1.8 public_key-1.7.2 reltool-0.8 runtime_tools-1.14.0.1 sasl-3.4.2 snmp-5.5.0.5 ssh-4.9.1.3 ssl-9.6.2.3 syntax_tools-2.2.1 tftp-1.0.2 tools-3.3.1.1 wx-1.9.0.1 xmerl-1.3.24 : OTP-22.3.4.19 : common_test-1.18.2.1 crypto-4.6.5.3 erl_interface-3.13.2.2 erts-10.7.2.11 megaco-3.18.8.4 odbc-2.12.4.1 snmp-5.5.0.5 wx-1.9.0.1 # asn1-5.0.12 compiler-7.5.4.3 debugger-4.2.8 dialyzer-4.1.1 diameter-2.2.3 edoc-0.11 eldap-1.2.8 erl_docgen-0.11 et-1.6.4 eunit-2.4.1 ftp-1.0.4.1 hipe-3.19.3 inets-7.1.3.3 jinterface-1.10.1 kernel-6.5.2.2 mnesia-4.16.3.1 observer-2.9.3 os_mon-2.5.1.1 parsetools-2.1.8 public_key-1.7.2 reltool-0.8 runtime_tools-1.14.0.1 sasl-3.4.2 ssh-4.9.1.3 ssl-9.6.2.3 stdlib-3.12.1 syntax_tools-2.2.1 tftp-1.0.2 tools-3.3.1.1 xmerl-1.3.24 : diff --git a/scripts/build-otp-tar b/scripts/build-otp-tar index 3f11026564..362d69d508 100755 --- a/scripts/build-otp-tar +++ b/scripts/build-otp-tar @@ -469,16 +469,6 @@ else echo " === Environment ==================================== " >> $build_log env >> $build_log - progress "Running autoconf in OTP" - echo " " >> $build_log - echo " === Running autoconf in OTP ================================ " >> $build_log - echo " " >> $build_log - echo "./otp_build autoconf" >> $build_log - ./otp_build autoconf >> $build_log 2>&1 - if [ $? -ne 0 ]; then - error "Failed to run autoconf in OTP" - fi - progress "Configuring OTP" echo " " >> $build_log echo " === Configuring OTP ================================ " >> $build_log diff --git a/system/doc/general_info/upcoming_incompatibilities.xml b/system/doc/general_info/upcoming_incompatibilities.xml index 02b9c35bda..69a0f6de16 100644 --- a/system/doc/general_info/upcoming_incompatibilities.xml +++ b/system/doc/general_info/upcoming_incompatibilities.xml @@ -92,5 +92,45 @@ </p> </section> + <section> + <marker id="atoms_be_utf8"/> + <title>Atoms will be encoded as UTF-8 by default</title> + <p> + As of OTP 26, the functions + <seemfa marker="erts:erlang#term_to_binary/1"> + <c>erlang:term_to_binary/1,2</c></seemfa> and + <seemfa marker="erts:erlang#term_to_iovec/1"> + <c>erlang:term_to_iovec/1,2</c></seemfa> will encode all atoms as + UTF-8 by default. The current default behavior is to encode atoms as + Latin-1 if possible. + </p> + <p> + If you implement your own decoding of the Erlang external format you + must either: + </p> + <list type="bulleted"> + <item> + <p> + Make sure your implementation supports the UTF-8 encodings + <seeguide marker="erts:erl_ext_dist#ATOM_UTF8_EXT"> + <c>ATOM_UTF8_EXT</c></seeguide> and + <seeguide marker="erts:erl_ext_dist#SMALL_ATOM_UTF8_EXT"> + <c>SMALL_ATOM_UTF8_EXT</c></seeguide>. + </p> + </item> + <item> + <p> + Call <seemfa marker="erts:erlang#term_to_binary/2"> + <c>erlang:term_to_binary/2</c></seemfa> or + <seemfa marker="erts:erlang#term_to_iovec/2"> + <c>erlang:term_to_iovec/2</c></seemfa> + with option <c>{minor_version,1}</c> to force Latin-1 encoding. This + is a more short-term solution as Latin-1 encoding may be phased out + and removed in later OTP releases. + </p> + </item> + </list> + </section> + </section> </chapter> diff --git a/system/doc/reference_manual/expressions.xml b/system/doc/reference_manual/expressions.xml index 2a2b7758e1..0392e1886d 100644 --- a/system/doc/reference_manual/expressions.xml +++ b/system/doc/reference_manual/expressions.xml @@ -1264,8 +1264,8 @@ Ei = Value | </pre> <p>Notice that bit string patterns cannot be nested.</p> <p>Notice also that "<c><![CDATA[B=<<1>>]]></c>" is interpreted as - "<c><![CDATA[B =<<1>>]]></c>" which is a syntax error. The correct way is - to write a space after '=': "<c><![CDATA[B= <<1>>]]></c>.</p> + "<c><![CDATA[B =< <1>>]]></c>" which is a syntax error. The correct way is + to write a space after '=': "<c><![CDATA[B = <<1>>]]></c>.</p> <p>More examples are provided in <seeguide marker="system/programming_examples:bit_syntax"> Programming Examples</seeguide>.</p> diff --git a/system/doc/reference_manual/ports.xml b/system/doc/reference_manual/ports.xml index 623bfe34e2..155569594d 100644 --- a/system/doc/reference_manual/ports.xml +++ b/system/doc/reference_manual/ports.xml @@ -131,7 +131,7 @@ <row> <cell align="left" valign="middle"><c>{Pid,{connect,NewPid}}</c></cell> <cell align="left" valign="middle">Sets the port owner of - <c>Port</c>to <c>NewPid</c>. Unless the port is already closed, + <c>Port</c> to <c>NewPid</c>. Unless the port is already closed, the port replies with<c>{Port,connected}</c> to the old port owner. Note that the old port owner is still linked to the port, but the new port owner is not.</cell> |