[ports] WIP: libwebp

[ports] WIP: libwebp

[Jens Staal]

[-- Attachment #1: Type: text/plain, Size: 482 bytes --]

Hi all,

I have been trying to get libwebp into ports as an optional dependency for
netsurf.

Unfortunately I have hit some compilation issues that I don't know how to solve
(I think it involves "##" substitutions which I remember was a problem before
when trying to compile perl too).
If anyone else is interested in getting libwebp support, I here share my current
progress on the port.

It is currently under a wip directory in ports, but would move to ape-libs when
it works.



[-- Attachment #2: wip_ports_libwebp.patch --]
[-- Type: text/plain, Size: 12042 bytes --]

# HG changeset patch
# User glenda@Krypton
# Date 1598752226 -7200
# Node ID a4b847abc7772090b01a12a4fd7c3aedafa5ba6a
# Parent  31104ca3b87bc43f919b4a6a206562eee4cecb51
Work-in-progress attempt at porting libwebp (optional dependency of netsurf).
Currently issues with compiling certain files (due to "##" substitution?), causing
"_body_last_cpuinfo" to be appended to some variable names.

diff -r 31104ca3b87b -r a4b847abc777 wip/libwebp/files/mkfile
--- /dev/null	Thu Jan 01 00:00:00 1970 +0000
+++ b/wip/libwebp/files/mkfile	Sun Aug 30 03:50:26 2020 +0200
@@ -0,0 +1,195 @@
+</$objtype/mkfile
+# I try to include all objects in one library instead of libwebp,
+# libwebpextras, libwebpdecoder libwebpmux libwebpdemux libimageio libimagenc
+LIB=/$objtype/lib/ape/libwebp.a
+OFILES=\
+#src/dec DEC_OBJS\
+	alpha_dec.$O\
+	buffer_dec.$O\
+	frame_dec.$O\
+	idec_dec.$O\
+	io_dec.$O\
+	quant_dec.$O\
+	tree_dec.$O\
+	vp8_dec.$O\
+	vp8l_dec.$O\
+	webp_dec.$O\
+#src/dsp DSP_DEC_OBJS\
+	alpha_processing.$O\
+	alpha_processing_mips_dsp_r2.$O\
+	alpha_processing_neon.$O\
+	alpha_processing_sse2.$O\
+	alpha_processing_sse41.$O\
+	cpu.$O\
+	dec.$O\
+	dec_clip_tables.$O\
+	dec_mips32.$O\
+	dec_mips_dsp_r2.$O\
+	dec_msa.$O\
+	dec_neon.$O\
+	dec_sse2.$O\
+	dec_sse41.$O\
+	filters.$O\
+	filters_mips_dsp_r2.$O\
+	filters_msa.$O\
+	filters_neon.$O\
+	filters_sse2.$O\
+	lossless.$O\
+	lossless_mips_dsp_r2.$O\
+	lossless_msa.$O\
+	lossless_neon.$O\
+	lossless_sse2.$O\
+	rescaler.$O\
+	rescaler_mips32.$O\
+	rescaler_mips_dsp_r2.$O\
+	rescaler_msa.$O\
+	rescaler_neon.$O\
+	rescaler_sse2.$O\
+	upsampling.$O\
+	upsampling_mips_dsp_r2.$O\
+	upsampling_msa.$O\
+	upsampling_neon.$O\
+	upsampling_sse2.$O\
+	upsampling_sse41.$O\
+    yuv.$O\
+    yuv_mips32.$O\
+    yuv_mips_dsp_r2.$O\
+    yuv_neon.$O\
+    yuv_sse2.$O\
+    yuv_sse41.$O\
+#src/dsp DSP_ENC_OBJS\
+	cost.$O\
+	cost_mips32.$O\
+	cost_mips_dsp_r2.$O\
+	cost_neon.$O\
+	cost_sse2.$O\
+	enc.$O\
+	enc_mips32.$O\
+	enc_mips_dsp_r2.$O\
+	enc_msa.$O\
+	enc_neon.$O\
+	enc_sse2.$O\
+	enc_sse41.$O\
+	lossless_enc.$O\
+	lossless_enc_mips32.$O\
+	lossless_enc_mips_dsp_r2.$O\
+	lossless_enc_msa.$O\
+	lossless_enc_neon.$O\
+	lossless_enc_sse2.$O\
+	lossless_enc_sse41.$O\
+	ssim.$O\
+	ssim_sse2.$O\
+#src/enc ENC_OBJS\
+	alpha_enc.$O\
+	analysis_enc.$O\
+	backward_references_cost_enc.$O\
+	backward_references_enc.$O\
+	config_enc.$O\
+	cost_enc.$O\
+	filter_enc.$O\
+	frame_enc.$O\
+	histogram_enc.$O\
+	iterator_enc.$O\
+	near_lossless_enc.$O\
+	picture_enc.$O\
+	picture_csp_enc.$O\
+	picture_psnr_enc.$O\
+	picture_rescale_enc.$O\
+	picture_tools_enc.$O\
+	predictor_enc.$O\
+	quant_enc.$O\
+	syntax_enc.$O\
+	token_enc.$O\
+	tree_enc.$O\
+	vp8l_enc.$O\
+	webp_enc.$O\
+#src/utils UTILS_DEC_OBJS\
+	bit_reader_utils.$O\
+	color_cache_utils.$O\
+	filters_utils.$O\
+	huffman_utils.$O\
+	quant_levels_dec_utils.$O\
+	random_utils.$O\
+	rescaler_utils.$O\
+	thread_utils.$O\
+	utils.$O\
+#src/utils UTILS_ENC_OBJS\
+	bit_writer_utils.$O\
+	huffman_encode_utils.$O\
+	quant_levels_utils.$O\
+#extras \
+	extras.$O\
+	quality_estimate.$O\
+#src/demux DEMUX_OBJS\
+	anim_decode.$O\
+	demux.$O\
+#src/mux MUX_OBJS\
+	anim_encode.$O\
+	muxedit.$O\
+	muxinternal.$O\
+	muxread.$O\
+#imageio IMAGE_UTIL_OBJS \
+	imageio_util.$O\
+#imageio EX_FORMAT_DEC_OBJS \
+	image_dec.$O\
+	jpegdec.$O\
+	metadata.$O\
+	pngdec.$O\
+	pnmdec.$O\
+	tiffdec.$O\
+	webpdec.$O
+
+HFILES=\
+	src/webp/decode.h \
+	src/webp/demux.h \
+	src/webp/encode.h \
+	src/webp/mux.h \
+	src/webp/mux_types.h \
+	src/webp/types.h \
+	src/webp/config.h
+
+</sys/src/cmd/mksyslib
+
+CC=pcc
+LD=pcc
+CFLAGS=-c -I. -I./src/webp -D_POSIX_SOURCE -D_SUSV2_SOURCE -D_BSD_EXTENSION -DHAVE_CONFIG_H -DWEBP_HAVE_JPEG -DWEBP_HAVE_PNG -DWEBP_HAVE_TIFF
+
+%.$O: src/dec/%.c
+	$CC $CFLAGS src/dec/$stem.c
+
+%.$O: src/demux/%.c
+	$CC $CFLAGS src/demux/$stem.c
+
+%.$O: src/dsp/%.c
+	$CC $CFLAGS src/dsp/$stem.c
+
+%.$O: src/enc/%.c
+	$CC $CFLAGS src/enc/$stem.c
+
+%.$O: src/mux/%.c
+	$CC $CFLAGS src/mux/$stem.c
+
+%.$O: src/utils/%.c
+	$CC $CFLAGS src/utils/$stem.c
+
+%.$O: extras/%.c
+	$CC $CFLAGS extras/$stem.c
+
+%.$O: imageio/%.c
+	$CC $CFLAGS imageio/$stem.c
+
+nuke:V:
+	mk clean
+	rm -f $LIB
+	rm -rf /sys/include/ape/webp
+
+install:V:
+	mkdir -p /sys/include/ape/webp
+	cp src/webp/decode.h /sys/include/ape/webp/
+	cp src/webp/demux.h /sys/include/ape/webp/
+	cp src/webp/encode.h /sys/include/ape/webp/
+	cp src/webp/mux.h /sys/include/ape/webp/
+	cp src/webp/mux_types.h /sys/include/ape/webp/
+	cp src/webp/types.h /sys/include/ape/webp/
+	
+
diff -r 31104ca3b87b -r a4b847abc777 wip/libwebp/files/src/demux/libwebpdemux.pc
--- /dev/null	Thu Jan 01 00:00:00 1970 +0000
+++ b/wip/libwebp/files/src/demux/libwebpdemux.pc	Sun Aug 30 03:50:26 2020 +0200
@@ -0,0 +1,11 @@
+prefix=/$objtype
+exec_prefix=${prefix}
+libdir=${exec_prefix}/lib/ape
+includedir=/sys/include/ape
+
+Name: libwebpdemux
+Description: Library for parsing the WebP graphics format container
+Version: 1.1.0
+Requires: libwebp >= 0.2.0
+Cflags: -I${includedir}
+Libs: -L${libdir} -lwebpdemux
diff -r 31104ca3b87b -r a4b847abc777 wip/libwebp/files/src/libwebp.pc
--- /dev/null	Thu Jan 01 00:00:00 1970 +0000
+++ b/wip/libwebp/files/src/libwebp.pc	Sun Aug 30 03:50:26 2020 +0200
@@ -0,0 +1,11 @@
+prefix=/$objtype
+exec_prefix=${prefix}/
+libdir=${exec_prefix}/lib/ape
+includedir=/sys/include/ape
+
+Name: libwebp
+Description: Library for the WebP graphics format
+Version: 1.1.0
+Cflags: -I${includedir}
+Libs: -L${libdir} -lwebp
+Libs.private: -lm  
diff -r 31104ca3b87b -r a4b847abc777 wip/libwebp/files/src/libwebpdecoder.pc
--- /dev/null	Thu Jan 01 00:00:00 1970 +0000
+++ b/wip/libwebp/files/src/libwebpdecoder.pc	Sun Aug 30 03:50:26 2020 +0200
@@ -0,0 +1,11 @@
+prefix=/$objtype
+exec_prefix=${prefix}
+libdir=${exec_prefix}/lib/ape
+includedir=/sys/include/ape
+
+Name: libwebpdecoder
+Description: Library for the WebP graphics format (decode only)
+Version: 1.1.0
+Cflags: -I${includedir}
+Libs: -L${libdir} -lwebpdecoder
+Libs.private: -lm  
diff -r 31104ca3b87b -r a4b847abc777 wip/libwebp/files/src/mux/libwebpmux.pc
--- /dev/null	Thu Jan 01 00:00:00 1970 +0000
+++ b/wip/libwebp/files/src/mux/libwebpmux.pc	Sun Aug 30 03:50:26 2020 +0200
@@ -0,0 +1,12 @@
+prefix=/$objtype
+exec_prefix=${prefix}
+libdir=${exec_prefix}/lib/ape
+includedir=/sys/include/ape
+
+Name: libwebpmux
+Description: Library for manipulating the WebP graphics format container
+Version: 1.1.0
+Requires: libwebp >= 0.2.0
+Cflags: -I${includedir}
+Libs: -L${libdir} -lwebpmux
+Libs.private: -lm
diff -r 31104ca3b87b -r a4b847abc777 wip/libwebp/files/src/webp/config.h
--- /dev/null	Thu Jan 01 00:00:00 1970 +0000
+++ b/wip/libwebp/files/src/webp/config.h	Sun Aug 30 03:50:26 2020 +0200
@@ -0,0 +1,150 @@
+/* src/webp/config.h.  Generated from config.h.in by configure.  */
+/* src/webp/config.h.in.  Generated from configure.ac by autoheader.  */
+
+/* Define if building universal (internal helper macro) */
+/* #undef AC_APPLE_UNIVERSAL_BUILD */
+
+/* Set to 1 if __builtin_bswap16 is available */
+/* #undef HAVE_BUILTIN_BSWAP16 */
+
+/* Set to 1 if __builtin_bswap32 is available */
+/* #undef HAVE_BUILTIN_BSWAP32 */
+
+/* Set to 1 if __builtin_bswap64 is available */
+/* #undef HAVE_BUILTIN_BSWAP64 */
+
+/* Define to 1 if you have the <cpu-features.h> header file. */
+/* #undef HAVE_CPU_FEATURES_H */
+
+/* Define to 1 if you have the <dlfcn.h> header file. */
+/* #undef HAVE_DLFCN_H */
+
+/* Define to 1 if you have the <GLUT/glut.h> header file. */
+/* #undef HAVE_GLUT_GLUT_H */
+
+/* Define to 1 if you have the <GL/glut.h> header file. */
+/* #undef HAVE_GL_GLUT_H */
+
+/* Define to 1 if you have the <inttypes.h> header file. */
+/* #undef HAVE_INTTYPES_H */
+
+/* Define to 1 if you have the <memory.h> header file. */
+/* #undef HAVE_MEMORY_H */
+
+/* Define to 1 if you have the <OpenGL/glut.h> header file. */
+/* #undef HAVE_OPENGL_GLUT_H */
+
+/* Have PTHREAD_PRIO_INHERIT. */
+/* #undef HAVE_PTHREAD_PRIO_INHERIT */
+
+/* Define to 1 if you have the <shlwapi.h> header file. */
+/* #undef HAVE_SHLWAPI_H */
+
+/* Define to 1 if you have the <stdint.h> header file. */
+#define HAVE_STDINT_H 1
+
+/* Define to 1 if you have the <stdlib.h> header file. */
+#define HAVE_STDLIB_H 1
+
+/* Define to 1 if you have the <strings.h> header file. */
+/* #undef HAVE_STRINGS_H */
+
+/* Define to 1 if you have the <string.h> header file. */
+#define HAVE_STRING_H 1
+
+/* Define to 1 if you have the <sys/stat.h> header file. */
+#define HAVE_SYS_STAT_H 1
+
+/* Define to 1 if you have the <sys/types.h> header file. */
+#define HAVE_SYS_TYPES_H 1
+
+/* Define to 1 if you have the <unistd.h> header file. */
+#define HAVE_UNISTD_H 1
+
+/* Define to 1 if you have the <wincodec.h> header file. */
+/* #undef HAVE_WINCODEC_H */
+
+/* Define to 1 if you have the <windows.h> header file. */
+/* #undef HAVE_WINDOWS_H */
+
+/* Define to the sub-directory where libtool stores uninstalled libraries. */
+#define LT_OBJDIR ".libs/"
+
+/* Name of package */
+#define PACKAGE "libwebp"
+
+/* Define to the address where bug reports for this package should be sent. */
+#define PACKAGE_BUGREPORT "https://bugs.chromium.org/p/webp"
+
+/* Define to the full name of this package. */
+#define PACKAGE_NAME "libwebp"
+
+/* Define to the full name and version of this package. */
+#define PACKAGE_STRING "libwebp 1.1.0"
+
+/* Define to the one symbol short name of this package. */
+#define PACKAGE_TARNAME "libwebp"
+
+/* Define to the home page for this package. */
+#define PACKAGE_URL "http://developers.google.com/speed/webp"
+
+/* Define to the version of this package. */
+#define PACKAGE_VERSION "1.1.0"
+
+/* Define to necessary symbol if this constant uses a non-standard name on
+   your system. */
+/* #undef PTHREAD_CREATE_JOINABLE */
+
+/* Define to 1 if you have the ANSI C header files. */
+#define STDC_HEADERS 1
+
+/* Version number of package */
+#define VERSION "1.1.0"
+
+/* Set to 1 if GIF library is installed */
+#define WEBP_HAVE_GIF 1
+
+/* Set to 1 if OpenGL is supported */
+/* #undef WEBP_HAVE_GL */
+
+/* Set to 1 if JPEG library is installed */
+#define WEBP_HAVE_JPEG 1
+
+/* Set to 1 if NEON is supported */
+/* #undef WEBP_HAVE_NEON */
+
+/* Set to 1 if runtime detection of NEON is enabled */
+/* #undef WEBP_HAVE_NEON_RTCD */
+
+/* Set to 1 if PNG library is installed */
+#define WEBP_HAVE_PNG 1
+
+/* Set to 1 if SDL library is installed */
+/* #undef WEBP_HAVE_SDL */
+
+/* Set to 1 if SSE2 is supported */
+/* #undef WEBP_HAVE_SSE2 */
+
+/* Set to 1 if SSE4.1 is supported */
+/* #undef WEBP_HAVE_SSE41 */
+
+/* Set to 1 if TIFF library is installed */
+#define WEBP_HAVE_TIFF 1
+
+/* Enable near lossless encoding */
+#define WEBP_NEAR_LOSSLESS 1
+
+/* Undefine this to disable thread support. */
+/* #undef WEBP_USE_THREAD */
+
+/* Define WORDS_BIGENDIAN to 1 if your processor stores words with the most
+   significant byte first (like Motorola and SPARC, unlike Intel). */
+#if defined AC_APPLE_UNIVERSAL_BUILD
+# if defined __BIG_ENDIAN__
+#  define WORDS_BIGENDIAN 1
+# endif
+#else
+# ifndef WORDS_BIGENDIAN
+/* #  undef WORDS_BIGENDIAN */
+# endif
+#endif
diff -r 31104ca3b87b -r a4b847abc777 wip/libwebp/manifest
--- /dev/null	Thu Jan 01 00:00:00 1970 +0000
+++ b/wip/libwebp/manifest	Sun Aug 30 03:50:26 2020 +0200
@@ -0,0 +1,2 @@
+98a052268cc4d5ece27f76572a7f50293f439c17a98e67c4ea0c7ed6f50ef043	libwebp-1.1.0.tar.gz
+
diff -r 31104ca3b87b -r a4b847abc777 wip/libwebp/mkfile
--- /dev/null	Thu Jan 01 00:00:00 1970 +0000
+++ b/wip/libwebp/mkfile	Sun Aug 30 03:50:26 2020 +0200
@@ -0,0 +1,15 @@
+</$objtype/mkfile
+
+OWNER='Jens Staal <staal1978@gmail.com>'
+DESCRIPTION='webp library for APE'
+VERSION=1.1.0
+DEPS=ape-libs/libtiff ape-libs/giflib ape-libs/libpng
+
+WEBSOURCE=https://storage.googleapis.com/downloads.webmproject.org/releases/webp/libwebp-$VERSION.tar.gz
+BUILDPATH=work/libwebp-$VERSION
+BUILDCOMMAND=mk
+INSTALLCOMMAND=mk install
+CLEANCOMMAND=mk clean
+UNINSTALLCOMMAND=mk nuke
+
+</sys/ports/Mk/mkfile

Re: [9front] [ports] WIP: libwebp

[ori]

> Hi all,
> 
> I have been trying to get libwebp into ports as an optional dependency for
> netsurf.
> 
> Unfortunately I have hit some compilation issues that I don't know how to solve
> (I think it involves "##" substitutions which I remember was a problem before
> when trying to compile perl too).
> If anyone else is interested in getting libwebp support, I here share my current
> progress on the port.
> 
> It is currently under a wip directory in ports, but would move to ape-libs when
> it works.

Thanks for taking a shot. I haven't had a chance to look yet, but
at the moment, there aren't any known bugs in our implementation
of '##' -- I think I made it work to spec. (There are some known
issues in other parts of cpp, some of which -- like the whitespace
handling around function-like macros -- are hard to fix.

Re: [9front] [ports] WIP: libwebp

[ori]

> Hi all,
> 
> I have been trying to get libwebp into ports as an optional dependency for
> netsurf.
> 
> Unfortunately I have hit some compilation issues that I don't know how to solve
> (I think it involves "##" substitutions which I remember was a problem before
> when trying to compile perl too).

In the maze of macros, we end up generating:

		static volatile VP8CPUInfo VP8DspInit_body_last_cpuinfo_used = 
			(VP8CPUInfo)& VP8DspInit_body_last_cpuinfo_used;

Which is an utterly stupid construction, but whatever.

Our compiler doesn't like static volatile. I think there's a simple fix,
since volatile doesn't actually affect the storage clasess:

diff -r cd0da90ae546 sys/src/cmd/cc/sub.c
--- a/sys/src/cmd/cc/sub.c	Sat Sep 12 17:40:41 2020 -0700
+++ b/sys/src/cmd/cc/sub.c	Sat Sep 12 20:21:49 2020 -0700
@@ -177,7 +177,7 @@
 {
 
 	b &= BCLASS;
-	switch(b) {
+	switch(b&~BVOLATILE) {
 	case 0:
 	case BREGISTER:
 		return CXXX;

Re: [9front] [ports] WIP: libwebp

[ori]

> diff -r cd0da90ae546 sys/src/cmd/cc/sub.c
> --- a/sys/src/cmd/cc/sub.c	Sat Sep 12 17:40:41 2020 -0700
> +++ b/sys/src/cmd/cc/sub.c	Sat Sep 12 20:21:49 2020 -0700
> @@ -177,7 +177,7 @@
>  {
>  
>  	b &= BCLASS;
> -	switch(b) {
> +	switch(b&~BVOLATILE) {
>  	case 0:
>  	case BREGISTER:
>  		return CXXX;

This fix doesn't do anything -- tested it on the wrong
file. Looks like the incorrect CLOCAL is slipping through
revertdcl. I can mash keys to make it work now, but I
don't understand the intent of the code in there, so
it'll take a bit more reading before I have soemthing
I'm comfortable committing.