process_vuxml.pl: When the parsing has finished, say so.

re: https://github.com/libexpat/libexpat/issues/928

Also, dispose() of the doc when completed.

Do not run if OFFLINE

exits, not exists

refer to $0, not the incorrect test-master-port.sh left over from copy/paste

Start using:

$FreshPorts::Config::OFFLINE

This replaces:
$FreshPorts::Config::ScriptDir/OFFLINE

Change bacula9 to bacula15

bacula9-client has been removed from the tree

use bacula15-client instead

Include host url

dump-package_imports.sh: allow for sorting by date and name (default)

re: https://github.com/FreshPorts/freshports/issues/593#issuecomment-2308384101

dump-package_imports.sh: order by repo_date

- the processed date is relevant
- But repo_date will be the same on all FreshPorts nodes

dump-package_imports.sh: add - re https://github.com/FreshPorts/freshports/issues/593

'Adjust for new package importing strategy'

'unlink the NewRepoImported signal, and Beacuse -> Because
'

report-notification-packages.pl: avoid looping in job-waiting.pl

Remove the flag which causes us to be launched.

Standardize all report emails.

Subject will be:

FreshPorts Notification: <topic>

Hopefully this will help people to filter easily.

Refer to add, not insert

And to package changes, not new packages.

Attempt to standardize on one name for emails:

FreshPorts notification

Add loop avoidance

After invoking UpdatePackagesFromRawPackages.py, always run report-notification-packages.pl

Add the report for package notifications.

re: https://github.com/FreshPorts/freshports/issues/542

It sends, duplicates, but it sends.

When reporting the number of found rows, make it easier for filtering

remove script for taking website offline - it is outdated

'refresh-abi logging improvements'

'refresh-abi logging improvements'

'remove some constants, improve abi refresh'

Let's not output anything by default. Let there be less.

Renove RefreshCachFileFlag - no longer used

re: https://github.com/FreshPorts/freshports/issues/58

While here, remove some items from show-config.pl which are no longer
present.

* DynamicDir
* FlagDir
* RefreshCachFileFlag
* WWWENPortsCategoriesFlag

'config.sh - add ABILOG; refresh-abi.sh - better logging'

add ABILOG

Remove this tag - going to replace it

Update PostgreSQL items

* For use by new abi scripts:
- Use export on PGSSLMODE
- Add PGSSLROOTCERT
- add DBUSER_ABI

add commit, and logging

add trailing ; to sql

Maintain the list of ABI

ABI get added, deleted. This set of scripts will maintain
that list in our abi table.

'-m'

Add missing logfatal()

Use grep -c

ls *.xml will result in ./system_status.sh: /bin/ls: Argument list too long

... when there are large numbers of commits.

Remove files/usr/src

This directory was used to decieve nvidia drivers into thinking we had
/usr/src installed.  re: https://github.com/FreshPorts/freshports/issues/152

'Use full path to git'

Use the full path to git

'removing our flag when we start processing. tell port->save() we are working on HEAD'

mention that process_default_versions.pl operates only upon head

Remove our signal

'cater for https://github.com/FreshPorts/freshports/issues/509'

'add categories_rebuild_search_html.sh'

'add categories_rebuild_search_html.sh'

'Include changes adding port_id to RemovePortFromCache() calls'

'improve cache invalidation for MOVED, UPDATING, etc'

'for UPDATING, use case insensitive for AFFECTS; trim rowcount on missing-port-categories.sh'

remove spaces from ROWCOUNT

Retire vuxml_ident.pl : used for subversion commits.

'add TEST_DEPENDS'

'refresh-ports.sh & process_updating.sh now log to DIRLOG, process_updating.pl looks for AFFECTS and avoids duplicates in cache_clearing_ports, add show-config.sh'

Remove code added for debugging

Fix UPDATING

With 20220623 and 20220620, 20220620 appears in column one.

That caused failures because of a faulty regex.

https://regex101.com to the rescue for this one

It was broken anyway, because the directory was wrong.


For cache clearing, we need to use the cache_clearing_ports table
The caching module is good only on webservers, not on ingress nodes,
which is where this script runs.

Althought this script is no longer relevant/used,
let's use the supplied configuration value for LOGDIR

Correct the ports tree file count

Reported by Laurent Chardon

'add code for make -V WWW'

'remove maildrop/mailfilter - no longer used now we're on git'

Drop the procmail, actually maildrop filter

Since FreeBSD moved to git, this is no longer required.

'Correct the link to commits

fixes https://github.com/FreshPorts/freshports/issues/335

reported by Matthew Seaman via private email
'

'add newlines to refresh-ports.pl'

add newlines to print statements

'add launch-refreshes.sh & long options for refresh-ports.pl'

Add in long options:

dryrun y/n
debug  y/n
limit  How many rows to process
offset Where to start int the result set

Stop using syslog, because output is now logged to a file

We shall keep distfiles around - no sense removing them if we are running multiple tests

add script name to syslog call

we shall assume git processing when refreshing

Create PORTSTOREFRESHFLAG flag

While here, derive BASEDIR_PACKAGER from BASEDIR

'move from ports-jail to jail'

'move from ports-jail to jail'

Move from chroot to jexec.

The ports-tree at ~freshports/ports-jail is no longer used.
Instead, we use one in a true proper jail.

For fortnightly, do 18 days, because it's not a perfect fortnight

Move to a /usr/ports location for the jail

Move from chroot to jexec

With make-pkg-message-in.sh, move back to apply-slist : it's great

'check ports_categories when not processing commits'

Add in some ignores with propset

Comment out the debugging code

column alignments

Report upon .errors files
Don't report upon ingress-svn any more

make-apply-slist.sh is not required in the port jail.

This functionality is carried out by get-list-of-current-categories.sh

Better shorter way to get the list of categories

'add get-list-of-current-categories.sh back in'

'-m'

Commit a work in progress, only to be deleted on my next commit

Remove 'svn commit checking status' since we don't do subversion commits any
more

With IgnoredItems being removed, this script is no longer required.

FreshPorts::Config::scriptpath and
FreshPorts::Config::ScriptDir have the same value. use only the latter.

Bring in the changes from https://github.com/FreshPorts/freshports/pull/261

'Previous release was mucked up. updated config.sh.sample'

'Previous release was the wrong tarball name - updated config.sh.sample'

A new iteration of the find

re https://github.com/FreshPorts/freshports/issues/245

Not all ports with a pkg-message have a pkg-message file.

Some have files/pkg-message.in, for example.

This script uses apply-slist to build that file.

stop counting .txt & start counting .xml

add DBUSER_REPORTING

'freebsd-cvs.sh: say if config.sh is not found
queue-status.pl: stop using the database, sort the queue keys so Nagios output is consistent, look only for files > 5 minutes old.
config.sh.sample: mention PORTSDIRBASE is best without a trailing slash
Add the monitoring directory, but I am not sure it is used yet.
process_vuxml.pl: improve spacing'

cate for OPTIONS_NAME

Pull in the vuxml code from the main branch, because we need the new vuxml
code here too.  While here, do some betting debugging.

'In /var/db/ingress/message-queues/incoming, we look for *.xml'

'better queue monitoring'

'add virtualhost-common-ssl.conf.sample'

Stop using FreshPorts::Utilities::CommitCountPeriod because we are counting
only file more than 5 minutes old.

adjust report_non_zero to cover all incoming and spooling directories. same for CountRecent

trailing / issues, add in src-stable-12

Check for and error out if config.sh is not found

'fix system_status.sh'

'email files are *.txt not, not *.txt.raw'

'Show reasons for update and number of updates'

'Adjust process_vuxml.pl to work on file open'

'For new security/vuxml/vuln.xml layout'

Move to using XML::DOM::XPath now that security/vuxml/vuln.xml is split into
several files. Proof of concept courtesy of bapt@FreeBSD.org

'first cut of the scripts for git'

load_xml_into_db.pl

* modified to check the source repo for this XML
* then invoke one of FreshPorts::xml_munge_git or FreshPorts::xml_munge_svn

load_xml_into_db_git.pl
load_xml_into_db_svn.pl

Modified to invoke the approviaate XML munging class:

* my $Munger = FreshPorts::XML_Munge_git->new($dbh);
* my $Munger = FreshPorts::XML_Munge_svn->new($dbh);

I started to modify these scripts to work with git, then realized they are
no longer used.

email and commit processing is handled by services:

[dan@devgit-ingress01:/usr/local/libexec/freshports-service] $ ls -l
total 26
-r-xr-xr-x  1 root  wheel  7267 Dec  6 22:17 freshports.sh
-r-xr-xr-x  1 root  wheel  1387 Dec  6 20:16 ingress.sh
-r-xr-xr-x  1 root  wheel  4109 Dec  6 22:13 ingress_svn.sh

Look there for invoking the xml processing scripts

We don't use this any more

Convert from `stuff` to $(stuff)

Convert from `stuff` to $(stuff)

Trim down the code by removing unnecessary items, such as the database
connection.

Let's just do a fetch, not a pull

add newline to end of file

spell quarterly

Add two functions needed for processing git commands.

Perhaps they should be in another file, but config.sh is included by thoese
that need these.

These two have been replaced by job-waiting.pl

Refer to master, not head.

We have jobs for both git and svn, and for ingress and freshports

Now that we have git and svn processing, create a separate script.

Make this work for the new repo locations and names

Remove unused $DaysRefreshed

Sort contents of %Jobs

Add $FreshPorts::Config::CheckGit => 'check_git.sh

Remove the usr files

re: https://github.com/FreshPorts/freshports/issues/152

The nvidia drivers are needlessly marked as IGNORE. For example https://www.freshports.org/x11/nvidia-driver-390/

The message is:

IGNORE: requires kernel source files in SRC_BASE=/usr/src

This message is produced by https://svnweb.freebsd.org/ports/head/Mk/Uses/kmod.mk?view=markup around line 25

Example:

[dan@dev-ingress01:/var/db/freshports/ports-jail/var/db/repos/PORTS-head/x11/nvidia-driver-390] $ make -V IGNORE
requires kernel source files in SRC_BASE=/usr/src
The code looks for ${SRC_BASE}/sys/Makefile

Creating that file avoids the issue:

$ sudo mkdir /usr/src/sys
$ sudo touch /usr/src/sys/Makefile
$ make -V IGNORE

$

Add in make-flavors-package-names.sh

extract PYTHON_PKGNAMEPREFIX from the port

Add the correct command to create the required thing

Create branch for git work

'Create a fake /usr/src/sys/Makefile to avoid spurious INGORE messages
by ports which use kmod
'

'fix a proctitle error with wide characters'

'remove debugging date'

'add report-maintainer-ports.pl'

Go case insensitive on the email address match

'add PGSSLMODE to config.sh'

'add BASEDIR_PACKAGER to config.sh'

'remove system() which did nothing in process_vuxml.pl'

Save recent work, before removing.

first version, going to start using -e next

Change our job-waiting loop.

When the job_waiting signal is set, run the waiting jobs and keep
runnint them until we don't find anything more to run.

'Always include the SQl when displaying an error'

'fix https://github.com/FreshPorts/freshports/issues/121'

Repair duplicate and extraneous results.

fixes https://github.com/FreshPorts/freshports/issues/121

Good script, but checkout-branch is better

Also refresh etc/make.conf in the PORT JAIL

'We have two scripts which check master-slave. One queries the databse (.pl), the other queries the Makefile (.sh). Fix both and comment appropriately'

Provide a script to copy the Jail scripts into the jail.

sysutils/bacula-server and sysutils/bacula-client are deprecated.

Use bacula9 instead

Check out the given branch to our local repo directory

Add hostname to test email

Adopt a new approach by putting all the directory queue locations into a
hash

'give the missing ports_categories email a better subject'

Add make-flavors-package-names.sh script

Add a way to refresh just one port. You must modify the code to specify that
port

Add a script to get:

* Number of queued commits
* Commit processing status

Delete this, created in error

Convert from using p5-Mail-Sender to p5-Email-Sender

* headers are now passed in via a hash

report-new-ports.pl now takes a range, as originally intended

show-config.pl: stop using email_client and use only email_server

Honor $FreshPorts::Config::ReportDebugging and stop using $Debug

Since this is now a periodic, and it's actually running now,
make it work since we need to send an email. The output goes to a log file
via:

$ grep output /etc/periodic.conf
hourly_output=/var/log/hourly.log
daily_output="/var/log/daily.log"
weekly_output="/var/log/weekly.log"
monthly_output="/var/log/monthly.log"

Change the default value for FRESHPORTS_JAIL_USER to freshports, from dan

Ignore config.sh

Better logging messages

All of the following deficiencies led to hiding permissions issues on a
directory.

* mention when process_vuxml.pl begins
* say vuxml_ident.pl, not vuxml ident
* say vuln_latest.pl, not vuxml latest
* toss the output of vuln_latest.pl into the log file
* if an error occurs during vuln_latest.pl, refer the reader to the logfile

Instead, now we see this in the vuxml.log file:

###
process_vuxml.pl finishes
'/var/db/freshports/cache/html' exists
trying to open '/var/db/freshports/cache/html/vuln-latest.html.tmp'
could not open '/var/db/freshports/cache/html/vuln-latest.html.tmp'
Tue Aug 14 14:59:55 UTC 2018 process_vuxml.sh vuxml finishes
####

'When things go wrong, log it'

Add a prefix so it's easy to filter via logcheck

When saying not found, mention not found by whom

Oops.  Didn't mean to do exactly that.