Commit Graph

275 Commits

Author SHA1 Message Date
Tianling Shen
3ab4809413 Merge Mainline
Signed-off-by: Tianling Shen <cnsztl@immortalwrt.org>
2022-04-14 10:34:23 +08:00
Tianling Shen
062547c724 scripts/download.pl: add CN mirrors for DEBIAN
Signed-off-by: Tianling Shen <cnsztl@immortalwrt.org>
(cherry picked from commit 0ea2f57158)
2022-04-14 01:07:38 +08:00
Daniel Golle
cf0dfc2dad scripts/gen_image_generic.sh: generate reproducible EFI filesystem
Generate FAT filesystem for EFI boot in a reproducible way:
 * use '--invariant' option of mkfs.fat
 * set timestamps of all files to SOURCE_DATE_EPOCH
 * make sure files are ordered locale-independent

Signed-off-by: Daniel Golle <daniel@makrotopia.org>
(cherry picked from commit aece8f5ae8)
2022-04-14 01:04:34 +08:00
Paul Spooren
5bf95283f9 build: store source_date_epoch as integer
The value is retreived from a env variable which defaults to be read as
a string. However the SOURCE_DATE_EPOCH is a unix timestamp aka integer.

Fix this to allow downstream tools to parse the value directly.

Signed-off-by: Paul Spooren <mail@aparcar.org>
(cherry picked from commit 2120cad38d)
2022-04-12 02:45:40 +08:00
Daniel Golle
e861b2a4dd scripts/gen_image_generic.sh: make ext4 bootfs reproducible
Set fixed timestamp for kernel other files in /boot filesystem.
This should help making x86 *combined* images reproducible.

Signed-off-by: Daniel Golle <daniel@makrotopia.org>
(cherry picked from commit 068ea2cde0)
2022-04-12 02:44:35 +08:00
Robert Marko
dc9747c292 scripts: ubinize-image: make rootfs optional
Currently ubinize-image script always expects the
rootfs image to be passed and a volume for it created.

So, to allow only ubinizing a kernel for example which
the MikroTik hAP ac3 and other new NAND devices from
MikroTik require make rootfs an optional parameter like
kernel.

Signed-off-by: Robert Marko <robimarko@gmail.com>
(cherry picked from commit 5a305e429f)
2022-04-10 21:07:40 +08:00
AmadeusGhost
233d89378a Merge Mainline 2022-04-08 11:35:35 +08:00
Paul Spooren
794eb1fe61 scripts: make sure conffiles are sorted
It may happen that conffiles are in different order on different builds.
Make sure they have the same order by sorting them.

FIX: #9612

Signed-off-by: Paul Spooren <mail@aparcar.org>
2022-04-04 17:20:29 +08:00
Tianling Shen
f99dff1f6a Merge Mainline
Signed-off-by: Tianling Shen <cnsztl@immortalwrt.org>
2022-03-26 22:51:06 +08:00
Tianling Shen
8e5662d5a3 scripts/download.pl: drop cqu mirror
Signed-off-by: Tianling Shen <cnsztl@immortalwrt.org>
(cherry picked from commit 7807fb0119)
2022-03-17 20:00:16 +08:00
Tianling Shen
1ff097db84 om-fwupgradecfg-gen: add missing catch for PA1200
Signed-off-by: Tianling Shen <cnsztl@immortalwrt.org>
2022-03-09 22:03:37 +08:00
Tianling Shen
e38cb49ef6 Merge Mainline
Signed-off-by: Tianling Shen <cnsztl@immortalwrt.org>
2022-03-08 11:43:38 +08:00
Georgi Valkov
c668329a78 mkhash: fix build errors on FreeBSD 13.0
be64enc, be16dec, and be32dec are declared on FreeBSD 13.0, in
/usr/include/sys/endian.h so we should not declare them.

Fixes the following error during feeds update:
staging_dir/host/bin/mkhash: No such file or directory

gcc scripts/mkhash.c
scripts/mkhash.c:111:1: error: redefinition of 'be64enc'
  111 | be64enc(void *buf, uint64_t u)

Signed-off-by: Georgi Valkov <gvalkov@abv.bg>
2022-03-06 23:56:37 +08:00
Tianling Shen
ec7b7129f4 Merge Mainline
Signed-off-by: Tianling Shen <cnsztl@immortalwrt.org>
2022-03-03 10:37:24 +08:00
Eneas U de Queiroz
d88e607184 scripts/diffconfig.sh: ensure config/conf is built
diffconfig.sh runs ./scripts/config/conf, but it does not get built
with 'make {menu,x,n}config.  Call 'make ./scripts/config/conf' to
ensure it's been built before running it, aborting in case of failure.

Signed-off-by: Eneas U de Queiroz <cotequeiroz@gmail.com>
Signed-off-by: Petr Štetiar <ynezz@true.cz>[removed Fixes: due revert]
2022-02-28 16:11:21 +08:00
Petr Štetiar
80e3983b36 Revert "scripts/diffconfig.sh: ensure config/conf is built"
This reverts commit c0849c1d9c as it seems
to introduce regression in config.buildinfo files produced at least by
buildbots:

 $ curl -s https://downloads.openwrt.org/snapshots/targets/ath79/generic/config.buildinfo | head -2
 make[3]: Entering directory '/builder/shared-workdir/build'
 make[3]: Leaving directory '/builder/shared-workdir/build'

References: https://github.com/openwrt/openwrt/issues/9297#issuecomment-1049719381
Signed-off-by: Petr Štetiar <ynezz@true.cz>
2022-02-26 11:16:43 +08:00
Tianling Shen
c96b15cc4d Merge Mainline
Signed-off-by: Tianling Shen <cnsztl@immortalwrt.org>
2022-02-25 13:03:01 +08:00
Eneas U de Queiroz
a8a70464fa scripts/diffconfig.sh: ensure config/conf is built
diffconfig.sh runs ./scripts/config/conf, but it does not get built
with 'make {menu,x,n}config.  Call 'make ./scripts/config/conf' to
ensure it's been built befpre running it.

Signed-off-by: Eneas U de Queiroz <cotequeiroz@gmail.com>
2022-02-24 11:36:25 +08:00
Eneas U de Queiroz
29cb743c4a build: scripts/config - update to kconfig-v5.14
Functional Changes
---------- -------
- make 'imply' not impose any restrictions: allow symbols implied by y
  to become m
- change "modules" from sub-option to first-level attribute

Bugfixes
--------
- nconf: fix core dump when searching in empty menu
- nconf: stop endless search loops
- xconfig: fix content of the main widget
- xconfig: fix support for the split view mode

Other Changes
----- -------
- highlight xconfig 'comment' lines with '***'
- xconfig: navigate menus on hyperlinks
- xconfig: drop support for Qt4
- improve host ncurses detection

Update the 'option modules' usage to just 'modules' in Config.in.

Signed-off-by: Eneas U de Queiroz <cotequeiroz@gmail.com>
2022-02-24 11:35:03 +08:00
Enrico Mioso
6faa43beae base-files: add eMMC sysupgrade support
Adds generic support for sysupgrading on eMMC-based devices.

Provide function emmc_do_upgrade and emmc_copy_config to be used in
/lib/upgrade/platform.sh instead of redundantly implementing the same
logic over and over again.
Similar to generic sysupgrade on NAND, use environment variables
CI_KERNPART, CI_ROOTPART and newly introduce CI_DATAPART to indicate
GPT partition names to be used. On devices with more than one MMC
block device, CI_ROOTDEV can be used to specify the MMC device for
partition name lookups.

Also allow to select block devices directly using EMMC_KERN_DEV,
EMMC_ROOT_DEV and EMMC_DATA_DEV, as using GPT partition names is not
always an option (e.g. when forced to use MBR).

To easily handle writing kernel and rootfs make use of sysupgrade.tar
format convention which is also already used for generic NAND support.

Signed-off-by: Enrico Mioso <mrkiko.rs@gmail.com>
Signed-off-by: Daniel Golle <daniel@makrotopia.org>
CC: Li Zhang <li.zhang@gl-inet.com>
CC: TruongSinh Tran-Nguyen <i@truongsinh.pro>
2022-02-12 12:26:37 +08:00
Daniel Golle
d669d1240a base-files: rename 'sdcard' to 'legacy-sdcard'
While an image layout based on MBR and 'bootfs' partition may be easy
to understand for users who are very used to the IBM PC and always have
the option to access the SD card outside of the device (and hence don't
really depend on other recovery methods or dual-boot), in my opinion
it's a dead end for many desirable features on embedded systems,
especially when managed remotely (and hence without an easy option to
access the SD card using another device in case things go wrong, for
example).

Let me explain:

* using a MSDOS/VFAT filesystem to store kernel(s) is problematic, as a
  single corruption of the bootfs can render the system into a state
  that it no longer boots at all. This makes dual-boot useless, or at
  least very tedious to setup with then 2 independent boot partitions
  to avoid the single point of failure on a "hot" block (the FAT index
  of the boot partition, written every time a file is changed in
  bootfs). And well: most targets even store the bootloader environment
  in a file in that very same FAT filesystem, hence it cannot be used
  to script a reliable dual-boot method (as loading the environment
  itself will already fail if the filesystem is corrupted).

* loading the kernel uImage from bootfs and using rootfs inside an
  additional partition means the bootloader can only validate the
  kernel -- if rootfs is broken or corrupted, this can lead to a reboot
  loop, which is often a quite costly thing to happen in terms of
  hardware lifetime.

* imitating MBR-boot behavior with a FAT-formatted bootfs partition
  (like IBM PC in the 80s and 90s) is just one of many choices on
  embedded targets. There are much better options with modern U-Boot
  (which is what we use and build from source for all targets booting
  off SD cards), see examples in mediatek/mt7622 and mediatek/mt7623.

Hence rename the 'sdcard' feature to 'legacy-sdcard', and prefix
functions with 'legacy_sdcard_' instead of 'sdcard_'.

Tested-by: Stijn Tintel <stijn@linux-ipv6.be>
Signed-off-by: Daniel Golle <daniel@makrotopia.org>
2022-02-12 12:26:08 +08:00
Tianling Shen
e9cabadf7c Merge Official Source
Signed-off-by: Tianling Shen <cnsztl@immortalwrt.org>
2022-01-03 13:54:53 +08:00
Paul Spooren
176e1fe191 scripts: dump-target-info print kernel versions
This commits adds the ability to print Kernel versions of all
targets/subtargets. If a testing Kernel is set print that version as
well.

Example output:

    apm821xx/nand 5.10
    apm821xx/sata 5.10
    arc770/generic 5.4
    archs38/generic 5.4
    armvirt/32 5.10
    armvirt/64 5.10
    at91/sam9x 5.10
    at91/sama5 5.10
    ath25/generic 5.4
    ath79/generic 5.4 5.10
    ath79/mikrotik 5.4 5.10
    --- %< ---

This should help to get a quick update on the state of Kernels.

Signed-off-by: Paul Spooren <mail@aparcar.org>
(cherry picked from commit 02de391b08)
2022-01-01 17:26:52 +08:00
AmadeusGhost
0c9ce5afbc Merge Mainline 2021-12-01 00:15:25 +08:00
Tianling Shen
d3c10b6224 scripts/download.pl: add new mirror
Signed-off-by: Tianling Shen <cnsztl@immortalwrt.org>
(cherry picked from commit 13c56d272d)
2021-11-30 20:44:49 +08:00
Tianling Shen
4c3a0af01c scripts/download.pl: tweak default mirrors
Signed-off-by: Tianling Shen <cnsztl@immortalwrt.org>
2021-11-29 13:23:45 +08:00
Tianling Shen
923d73b691 scripts/download.pl: reorder self-hosted mirrors
Signed-off-by: Tianling Shen <cnsztl@immortalwrt.org>
(cherry picked from commit d26b4ff4de)
2021-11-29 03:31:03 +08:00
Tianling Shen
404f8f2992 scripts/download.pl: add immortalwrt's mirrors
Signed-off-by: Tianling Shen <cnsztl@immortalwrt.org>
2021-11-21 01:07:21 +08:00
Felix Fietkau
cb8dc7b14e toolchain: add a version that can be bumped to force toolchain/target rebuild
This can be used to ensure that the compiled code is up to date, when
something important changes in the toolchain.
A recent example of this is the gcc 11 fix for a code miscompilation issue

Signed-off-by: Felix Fietkau <nbd@nbd.name>
2021-11-19 23:58:57 +08:00
Tianling Shen
45ff0ed571 Merge Mainline
Signed-off-by: Tianling Shen <cnsztl@immortalwrt.org>
2021-11-11 20:20:39 +08:00
Daniel Golle
aa58b28e47 base-files, metadata: support additional group membership
Some packages may require additional group membership for the system
user added by that package. Allow defining additional groups as third
member of the ':'-separated tuple, allowing to specify multiple
','-separated groups with optional GID.

Example:
USERID:=foouser=1000:foogroup=1000:addg1=1001,addg2=1002,addg3

Signed-off-by: Daniel Golle <daniel@makrotopia.org>
(cherry picked from commit b2aca61360)
2021-11-10 15:42:57 +08:00
Daniel Golle
850b6ea0d4 base-files, metadata: support additional group membership
Some packages may require additional group membership for the system
user added by that package. Allow defining additional groups as third
member of the ':'-separated tuple, allowing to specify multiple
','-separated groups with optional GID.

Example:
USERID:=foouser=1000:foogroup=1000:addg1=1001,addg2=1002,addg3

Signed-off-by: Daniel Golle <daniel@makrotopia.org>
2021-11-03 23:58:28 +08:00
Josh Soref
e72185a4a1 scripts: fix various typos
This only affects typos in comments or user-facing output.

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
[only picks changes to scripts, drop "commandline" replacement,
 fix case for "arbitrary", improve commit message]
Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2021-11-02 23:49:21 +08:00
Jan-Niklas Burfeind
cffc1cd77b scripts: eva_ramboot.py: remove unused import
concludes:
commit e7bc8984d9 ("scripts: make eva_ramboot.py offset configurable")

Signed-off-by: Jan-Niklas Burfeind <git@aiyionpri.me>
2021-11-02 23:46:05 +08:00
Tianling Shen
f4effdee24 Merge Mainline
Signed-off-by: Tianling Shen <cnsztl@immortalwrt.org>
2021-10-27 12:18:41 +08:00
Felix Fietkau
3ae00183b0 scripts/rstrip.sh: do not strip .o files with STRIP_KMOD
Fixes packaging of eBPF modules

Signed-off-by: Felix Fietkau <nbd@nbd.name>
2021-10-15 23:56:40 +08:00
Petr Štetiar
dc89ad3dfe scripts/gen_image_generic.sh: fix more shellcheck warnings
Fixes following shellcheck warnings:

 In scripts/gen_image_generic.sh line 20:
 cyl=$(( (KERNELSIZE + ROOTFSSIZE) * 1024 * 1024 / (head * sect * 512)))
 ^-^ SC2034: cyl appears unused. Verify use (or export if used externally).
 --
 In scripts/gen_image_generic.sh line 34:
     [ -n "$PADDING" ] && dd if=/dev/zero of="$OUTPUT" bs=512 seek="$(($ROOTFSOFFSET + $ROOTFSSIZE))" conv=notrunc count="$sect"
                                                                       ^-----------^ SC2004: $/${} is unnecessary on arithmetic variables.
 --
 In scripts/gen_image_generic.sh line 35:
     mkfs.fat -n kernel -C "$OUTPUT.kernel" -S 512 "$(($KERNELSIZE / 1024))"
                                                       ^---------^ SC2004: $/${} is unnecessary on arithmetic variables.

Signed-off-by: Petr Štetiar <ynezz@true.cz>
(cherry picked from commit 92d7cedcad)
2021-10-14 21:25:07 +08:00
Paul Spooren
7218521845 scripts: package-metadata add pkgmanifestjson call
The new `pkgmanifestjson` call prints all package manifest of a feed in
JSON format. This function can be used to print an overview of packages
information used for downstream tooling.

The script is entirely based on Petrs work on dependency visualisation.

Signed-off-by: Paul Spooren <mail@aparcar.org>
2021-10-12 23:56:16 +08:00
Paul Spooren
736c627160 scripts: store maintainer in package metadata
The maintainer could be usable for downstream tooling, so start storing
it in the metadata.

Signed-off-by: Paul Spooren <mail@aparcar.org>
2021-10-12 23:55:51 +08:00
Damien Mascord
549f6e6b93 scripts: mkits.sh: Allow legacy @ mode for dts creation
commit 5ec60cbe9d ("scripts: mkits.sh: replace @ with - in nodes")
broke support for Meraki MR32 and this patch makes the replacement
configurable allowing for specifying the @ or - or whatever character
that is desired to retain backwards compatibility with existing devices.

For example, this patch includes the fix for the Meraki MR32 in
target/linux/bcm53xx/image for meraki_mr32:

  DEVICE_DTS_DELIMITER := @
  DEVICE_DTS_CONFIG := config@1

Fixes: 5ec60cbe9d ("scripts: mkits.sh: replace @ with - in nodes")
Signed-off-by: Damien Mascord <tusker@tusker.org>
[Added tags, checkpatch.pl fixes, noted that this is for old stuff]
Signed-off-by: Christian Lamparter <chunkeey@gmail.com>
2021-10-12 23:54:34 +08:00
Ansuel Smith
2969a245ec scripts: handle gcc and linux in dl_cleanup script
Handle gcc and linux with a special regex that set their progname with
their major version. This way every minor version can be cleared. The
build cleanup logic can be tweaked later to clean the entire toolchain
and target dir with a different gcc version.

Signed-off-by: Ansuel Smith <ansuelsmth@gmail.com>
[reformat commit message]
Signed-off-by: Paul Spooren <mail@aparcar.org>
2021-10-12 23:51:27 +08:00
Ansuel Smith
7b630b974e scripts: format dl_cleanup to black format python style
Forma dl_cleanup python script to black style.

Signed-off-by: Ansuel Smith <ansuelsmth@gmail.com>
2021-10-12 23:51:14 +08:00
Ansuel Smith
6477cae7a1 scripts: add missing regex for dl_cleanup script
Regex xxx-YYYY-MM-DD-GIT_SHASUM was missing. Add the new regex to improve
and better find outdated package. This also fix a bug where some bug were
incorrectly detected as packagename-yyyy-mm-dd instead of packagename due
to them be parsed by the wrong parser

Example:

    openwrt-keyring-2021-02-20-49283916.tar.xz

Signed-off-by: Ansuel Smith <ansuelsmth@gmail.com>
[added example in commit message]
Signed-off-by: Paul Spooren <mail@aparcar.org>
2021-10-12 23:50:56 +08:00
Tianling Shen
11805f2f70 Merge Mainline
Signed-off-by: Tianling Shen <cnsztl@immortalwrt.org>
2021-09-23 10:13:31 +08:00
Paul Spooren
e9cb0a7753 build: store artifacts in JSON
Multiple profiles create artifacts, these should be stored in the JSON
file as well, allowing downstream tooling to show those files, too.

Artifacts don't have specific filesystems so only the fields `name`,
`type` and `sha256` are available.

Rename env variable names from IMAGE_ to FILE_ prefixes to reflect that
images, kernels and artifacts are added with the same command.

Signed-off-by: Paul Spooren <mail@aparcar.org>
(cherry picked from commit 5ef4608c02)
2021-09-17 04:49:04 +08:00
Adrian Schmutzler
7984a95946 treewide: rename IMAGE_PREFIX/IMAGE_NAME to DEVICE_IMG_*
We so far had two variables IMG_PREFIX and IMAGE_PREFIX with
different content. Since these names are obviously quite
confusing, this patch renames the latter to DEVICE_IMG_PREFIX,
as it's a device-dependent variable, while IMG_PREFIX is only
(sub)target-dependent.

For consistency, also rename IMAGE_NAME to DEVICE_IMG_NAME, as
that's a device-dependent variable as well.

Cc: Paul Spooren <mail@aparcar.org>
Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
(cherry picked from commit ef2cb8572b)
2021-09-17 04:48:26 +08:00
Tianling Shen
f895287396 Merge Official Source
Signed-off-by: Tianling Shen <cnsztl@immortalwrt.org>
2021-09-03 03:26:54 +08:00
SuLingGG
e45c211c5a scripts/download.pl: update openwrt.cc mirror url
[updated url]
Signed-off-by: Tianling Shen <cnsztl@immortalwrt.org>
(cherry picked from commit 009c172171)
2021-08-30 23:07:46 +08:00
Stijn Tintel
4237ba7d6f base-files: add generic sdcard upgrade method
Add a generic sdcard upgrade method instead of duplicating code in yet
another target, and add a feature flag to only install this upgrade
method in targets that set this flag. Copied from mvebu.

Signed-off-by: Stijn Tintel <stijn@linux-ipv6.be>
2021-08-07 12:09:16 +08:00
Stijn Tintel
bc335784f0 scripts/target-metadata.pl: order features alphabetically
We generaly do this in many other places, so let's do it here also.

Signed-off-by: Stijn Tintel <stijn@linux-ipv6.be>
2021-08-07 12:08:59 +08:00