Skip to content
This repository was archived by the owner on Nov 26, 2024. It is now read-only.

Commit 3130fd2

Browse files
authored
Cache file (#8)
Replaced SQLite with YAML file cache
1 parent b27a348 commit 3130fd2

11 files changed

Lines changed: 154 additions & 189 deletions

File tree

README.md

Lines changed: 27 additions & 23 deletions
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,7 @@ This tool parses Go binary dependencies and calls [NVD database](https://nvd.nis
1010
4. [Cache](#cache)
1111
- [Memcachier](#memcachier)
1212
- [Memcached](#memcached)
13-
- [SQLite](#sqlite)
13+
- [File](#file)
1414
5. [Versions](#versions)
1515
6. [How to Fix Vulnerabilities](#how-to-fix-vulnerabilities)
1616
7. [Information about vulnerabilities](#information-about-vulnerabilities)
@@ -93,6 +93,9 @@ memcachier:
9393
memcached:
9494
address: 127.0.0.1:11211
9595
expiration: 86400
96+
file:
97+
name: ~/.gobinsec-cache.yml
98+
expiration: 86400
9699
ignore:
97100
- "CVE-2020-14040"
98101
```
@@ -103,6 +106,7 @@ Configuration fields are the following:
103106
- **strict**: tells if we should consider vulnerability matches without version as matching dependency
104107
- **memcachier** is the configuration for *memcachier*, with **address**, **expiration** (time in seconds), **username** and **password**
105108
- **memcached** is the configuration for *memcached*, with **address** and **expiration** time in seconds
109+
- **file** is the configuration for *file* cache, with **name** and **expiration** time in seconds
106110
- **ignore**: a list of CVE vulnerabilities to ignore
107111

108112
You can also set NVD API Key in your environment with variable *NVD_API_KEY*. This key may be overwritten with value in configuration file. Your API key must be set in environment to be able to run integration tests (with target *integ*).
@@ -113,55 +117,55 @@ Note that without API key, you will be limited to *10* requests in a rolling *60
113117

114118
A cache is useful because if you perform more call to NVD database than allowed, your calls will significantly slow down. Gobinsec tries to build caches in this order:
115119

116-
### Memcached
120+
### Memcachier
117121

118-
A cache is built with *Memcached* if following section is found in configuration file:
122+
A cache is built with *Memcachier* if following section is found in configuration file:
119123

120124
```yaml
121-
memcached:
125+
memcachier:
122126
address: ...
123127
expiration: ...
128+
username: ...
129+
password: ...
124130
```
125131
126-
Else it will look for following environment variables:
132+
Else, il will look for following environment variables:
127133
128134
```
129-
MEMCACHED_ADDRESS
130-
MEMCACHED_EXPIRATION
135+
MEMCACHIER_ADDRESS
136+
MEMCACHIER_EXPIRATION
137+
MEMCACHIER_USERNAME
138+
MEMCACHIER_PASSWORD
131139
```
132140

133-
A sample [docker-compose.yml](https://github.com/intercloud/gobinsec/blob/main/docker-compose.yml) file to start a *memcached* instance is provided in this project.
141+
[Memcachier](https://www.memcachier.com) is an online cache provider with free tiers.
134142

135-
### Memcachier
143+
### Memcached
136144

137-
A cache is built with *Memcachier* if following section is found in configuration file:
145+
A cache is built with *Memcached* if following section is found in configuration file:
138146

139147
```yaml
140-
memcachier:
148+
memcached:
141149
address: ...
142150
expiration: ...
143-
username: ...
144-
password: ...
145151
```
146152
147-
Else, il will look for following environment variables:
153+
Else it will look for following environment variables:
148154
149155
```
150-
MEMCACHIER_ADDRESS
151-
MEMCACHIER_EXPIRATION
152-
MEMCACHIER_USERNAME
153-
MEMCACHIER_PASSWORD
156+
MEMCACHED_ADDRESS
157+
MEMCACHED_EXPIRATION
154158
```
155159

156-
[Memcachier](https://www.memcachier.com) is an online cache provider with free tiers.
160+
A sample [docker-compose.yml](https://github.com/intercloud/gobinsec/blob/main/docker-compose.yml) file to start a *memcached* instance is provided in this project.
157161

158-
### SQLite
162+
### File
159163

160-
If none of preceding configuration is found in configuration and none of related environment variables, *Gobinsec* will use *SQLite* for caching. By default, database file is stored in *~/.gobinsec.db* and cache duration is of one day (or *86400* seconds). You can overwrite these default values with following configuration section:
164+
If none of preceding configuration is found in configuration and none of related environment variables, *Gobinsec* will use *YAML* file for caching. By default, database file is stored in *~/.gobinsec-cache.yml* and cache duration is of one day (or *86400* seconds). You can overwrite these default values with following configuration section:
161165

162166
```yaml
163-
sqlite:
164-
file: "/path/to/database.db"
167+
file:
168+
name: "/path/to/file.yml"
165169
expiration: 86400
166170
```
167171

go.mod

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,6 @@ go 1.18
55
require (
66
github.com/bradfitz/gomemcache v0.0.0-20220106215444-fb4bf637b56d
77
github.com/fatih/color v1.13.0
8-
github.com/mattn/go-sqlite3 v1.14.12
98
github.com/memcachier/gomemcache v0.0.0-20170425125614-d027381f7653
109
gopkg.in/yaml.v3 v3.0.0-20210107192922-496545a6307b
1110
)

go.sum

Lines changed: 0 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -8,8 +8,6 @@ github.com/mattn/go-colorable v0.1.12/go.mod h1:u5H1YNBxpqRaxsYJYSkiCWKzEfiAb1Gb
88
github.com/mattn/go-isatty v0.0.12/go.mod h1:cbi8OIDigv2wuxKPP5vlRcQ1OAZbq2CE4Kysco4FUpU=
99
github.com/mattn/go-isatty v0.0.14 h1:yVuAays6BHfxijgZPzw+3Zlu5yQgKGP2/hcQbHb7S9Y=
1010
github.com/mattn/go-isatty v0.0.14/go.mod h1:7GGIvUiUoEMVVmxf/4nioHXj79iQHKdU27kJ6hsGG94=
11-
github.com/mattn/go-sqlite3 v1.14.12 h1:TJ1bhYJPV44phC+IMu1u2K/i5RriLTPe+yc68XDJ1Z0=
12-
github.com/mattn/go-sqlite3 v1.14.12/go.mod h1:NyWgC/yNuGj7Q9rpYnZvas74GogHl5/Z4A/KQRfk6bU=
1311
github.com/memcachier/gomemcache v0.0.0-20170425125614-d027381f7653 h1:222emoxOt/bCmNHp8Xt0Pr5Am3gIbqRKFpb4CQ9O2SI=
1412
github.com/memcachier/gomemcache v0.0.0-20170425125614-d027381f7653/go.mod h1:KoYVbOQexD45AOLfn+gsFB6c3o4ANzP1QKzjE6tZbK0=
1513
golang.org/x/sys v0.0.0-20200116001909-b77594299b42/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=

gobinsec/cache-file.go

Lines changed: 111 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,111 @@
1+
package gobinsec
2+
3+
import (
4+
"os"
5+
"path/filepath"
6+
"strings"
7+
"time"
8+
9+
"gopkg.in/yaml.v3"
10+
)
11+
12+
// FileConfig is the configuration for file cache
13+
type FileConfig struct {
14+
File string `yaml:"name"`
15+
Expiration int32 `yaml:"expiration"`
16+
}
17+
18+
// NewFileConfig builds configuration for file cache
19+
func NewFileConfig(config *FileConfig) *FileConfig {
20+
if config == nil {
21+
config = &FileConfig{}
22+
}
23+
if config.File == "" {
24+
config.File = "~/.gobinsec-cache.yml"
25+
}
26+
if strings.HasPrefix(config.File, "~/") {
27+
home, err := os.UserHomeDir()
28+
if err != nil {
29+
return nil
30+
}
31+
config.File = filepath.Join(home, config.File[2:])
32+
}
33+
if config.Expiration == 0 {
34+
config.Expiration = 86400
35+
}
36+
return config
37+
}
38+
39+
// DependencyCache is an entry of dependency cache
40+
type DependencyCache struct {
41+
Date string
42+
Vulnerabilities string
43+
}
44+
45+
// FileCache is the cache instance
46+
type FileCache struct {
47+
File string
48+
Expiration int32
49+
Cache map[string]DependencyCache
50+
}
51+
52+
// NewFileCache builds a cache using file
53+
func NewFileCache(config *FileConfig) Cache {
54+
cache := make(map[string]DependencyCache)
55+
file, err := os.ReadFile(config.File)
56+
if err == nil {
57+
if err := yaml.Unmarshal(file, cache); err != nil {
58+
cache = make(map[string]DependencyCache)
59+
}
60+
}
61+
fileCache := &FileCache{
62+
File: config.File,
63+
Expiration: config.Expiration,
64+
Cache: cache,
65+
}
66+
fileCache.CleanCache()
67+
return fileCache
68+
}
69+
70+
// Get returns NVD response for given dependency
71+
func (fc *FileCache) Get(d *Dependency) []byte {
72+
dependency, ok := fc.Cache[d.Name]
73+
if ok {
74+
return []byte(dependency.Vulnerabilities)
75+
}
76+
return nil
77+
}
78+
79+
// Set put NVD response for given dependency in cache
80+
func (fc *FileCache) Set(d *Dependency, v []byte) {
81+
now := time.Now().UTC().Format("2006-01-02T15:04:05")
82+
cache := DependencyCache{
83+
Date: now,
84+
Vulnerabilities: string(v),
85+
}
86+
fc.Cache[d.Name] = cache
87+
}
88+
89+
// Ping does nothing
90+
func (fc *FileCache) Open() error {
91+
return nil
92+
}
93+
94+
// Clean saves file
95+
func (fc *FileCache) Close() {
96+
text, err := yaml.Marshal(fc.Cache)
97+
if err == nil {
98+
os.WriteFile(fc.File, text, 0644) // nolint:errcheck,gosec,gomnd // if error writing cache, do nothing
99+
}
100+
}
101+
102+
// CleanCache removes obsolete cache entries
103+
func (fc *FileCache) CleanCache() {
104+
duration := time.Duration(fc.Expiration) * time.Second
105+
limit := time.Now().UTC().Add(-duration).Format("2006-01-02T15:04:05")
106+
for name, cache := range fc.Cache {
107+
if cache.Date < limit {
108+
delete(fc.Cache, name)
109+
}
110+
}
111+
}

gobinsec/cache-memcached.go

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -70,10 +70,10 @@ func (mc *MemcachedClient) Set(d *Dependency, v []byte) {
7070
}
7171

7272
// Ping calls memcached
73-
func (mc *MemcachedClient) Ping() error {
73+
func (mc *MemcachedClient) Open() error {
7474
return mc.Client.Ping()
7575
}
7676

7777
// Clean does nothing
78-
func (mc *MemcachedClient) Clean() {
78+
func (mc *MemcachedClient) Close() {
7979
}

gobinsec/cache-memcachier.go

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -78,7 +78,7 @@ func (mc *MemcachierClient) Set(d *Dependency, v []byte) {
7878
}
7979

8080
// Ping calls Memcachier
81-
func (mc *MemcachierClient) Ping() error {
81+
func (mc *MemcachierClient) Open() error {
8282
item := memcache.Item{
8383
Key: "test",
8484
Value: []byte("test"),
@@ -88,5 +88,5 @@ func (mc *MemcachierClient) Ping() error {
8888
}
8989

9090
// Clean does nothing
91-
func (mc *MemcachierClient) Clean() {
91+
func (mc *MemcachierClient) Close() {
9292
}

0 commit comments

Comments
 (0)